orthrus.sparse.feature_selection namespace¶

Submodules¶

orthrus.sparse.feature_selection.IterativeFeatureRemoval module¶

class orthrus.sparse.feature_selection.IterativeFeatureRemoval.IFR(classifier=None, scorer=<function balanced_accuracy_score>, weights_handle: str = 'weights_', repetition: int = 10, partition_method: str = 'stratified_k-fold', nfolds: int = 3, max_iters: int = 5, cutoff: float = 0.75, jumpratio: float = 100.0, max_features_per_iter_ratio: float = 0.8, verbosity: int = 0, verbose_frequency: int = 10, num_cpus_per_worker: float = 1.0, num_gpus_per_worker: float = 0.0, local_mode=False)¶

Bases: object

The Iterative Feature Removal algorithms extracts features for many data partitions independently, and combines them and keep track of feature frequency, weights and iterations in which each feature was extracted. During execution, the data is partitioned into training and validation sets ‘repetition’ number of times and then for each partition one feature set is extracted. So, a total of repetition * num_partitions independent feature sets are extracted and the results are merged to create the output. These independent feature set extractions are batched and run in parallel using the ray package. Each feature extraction is a ray worker, see below to check how to specify resource requirements for each worker.

For each feature set, the algorithm can halt because of the following conditions:

Score on validation partition is below cutoff

Jump does not occur in the array of sorted absolute weights

Jump occurs but the weight at the jump is too small ( < 10e-6)

Number of features selected for the current iteration is greater than max_features_per_iter_ratio * num_samples in training partition. This condition prevents overfitting.

max_iters number of iterations complete successfully

When one of these conditions happen, further feature extraction on the current fold is stopped.

Parameters

classifier (object) – Classifier to run the classification experiment with; must have the sklearn equivalent of a fit and predict method. Default classifier is orthrus.sparse.classifiers.svm .SSVMClassifier, it will be a CPU based classifier if num_gpus_per_worker is 0, otherwise it will be a GPU classifier.
scorer (object) – Function which scores the prediction labels on training and test partitions. This function should accept two arguments: truth labels and prediction labels. This function should output a score between 0 and 1 which can be thought of as an accuracy measure. See sklearn.metrics.balanced_accuracy_score for an example.
weights_handle (str) – Name of classifier attribute containing feature weights. Default is ‘weights_’.
repetition (int) – Determines the number of times to partition the dataset. (default: 10)
partition_method (string) – A partition method that is compatible with calcom.utils.generate_partitions (default: ‘stratified_k-fold’)
nfolds (int) – The number of folds to partition data into (default: 3)
max_iters (int) – Determines the maximum number of iterations of IFR on one data partition(default: 5)
cutoff (float) – Threshold for the validation score to halt the process. (default: 0.75)
jumpratio (float) – The relative drop in the magnitude of coefficients in weight vector to identify numerically zero weights (default: 100)
max_features_per_iter_ratio (float) – A fraction that limits the max number of features that can be extracted per iteration. (default: 0.8) if the number if selected features is greater than max_features_per_iter_ratio * #samples in training partition, further execution on the current fold is stopped.
verbosity (int) – Determines verbosity of print statments; 0 for no output; 2 for full output. (default: 0)
verbose_frequency (int) – this parameter controls the frequency of progress outputs for the ray workers to console; an output is printed to console after every verbose_frequency number of processes complete execution. (default: 10)
num_cpus_per_worker (float) – Number of CPUs each worker needs. This can be a fraction, check ray specifying required resources for more details. (default: 1.)
num_gpus_per_worker (float) –
Number of GPUs each worker needs. This can be fraction, check ray specifying required resources for more details. (default: 0.)

diagnostic_information_¶

Holds execution the following information for each interation of each partition. ‘train_scores’ (list) : Each element is a list of training scores for the feature selection on one data partatition,

the number of elements in this inner list is the number of iterations IFR ran for, for this particular data partition.

‘validation_scores’ (list): Each element is a list of test scores for the feature selection on one data partatition,

the number of elements in this inner list is the number of iterations IFR ran for, for this particular data partition.

‘sorted_abs_weights (list)’: Each element is a list of sorted absolute weights for the classifier for the feature selection on one data partatition,

the number of elements in this inner list is the number of iterations IFR ran for, for this particular data partition.

‘weight_ratios’ (list)’: Each element is a list of weight ratios for the classifier for the feature selection on one data partatition,

the number of elements in this inner list is the number of iterations IFR ran for, for this particular data partition.

‘features’ (list)’: Each element is a list of selected feature ids for the feature selection on one data partatition,

the number of elements in this inner list is the number of iterations IFR ran for, for this particular data partition.

‘true_feature_count’ (list): Each element is a list of true number of features that IFR determined were to be selected for the feature selection on one data partatition,

the number of elements in this inner list is the number of iterations IFR ran for, for this particular data partition.

‘exit_reasons’ (list): Each element contains the reason for why the IFR stopped for the feature selection on one data partatition. These are one of the following reasons:

exception_in_model_fitting
validation_score_cutoff: Score on validation partition is below cutoff
jump_failed: Jump does not occur in the array of sorted absolute weights
small_weight_at_jump: Jump occurs but the weight at the jump is too small ( < 10e-6)
max_features_per_iter_breached: Number of features selected for the current iteration is greater than max_features_per_iter_ratio * num_samples in training partition. This condition prevents overfitting.
max_iters: max_iters number of iterations complete successfully

Type: dict

Examples

>>> import orthrus.core.dataset as DS
>>> import orthrus.sparse.feature_selection.IterativeFeatureRemoval as IFR
>>> x = DS.load_dataset('path/to/gse_730732.h5')
>>> from calcom.solvers import LPPrimalDualPy
>>> import calcom
>>> model = calcom.classifiers.SSVMClassifier()
>>> model.params['C'] = 1.
>>> model.params['method'] = LPPrimalDualPy
>>> model.params['use_cuda'] = True
>>> weights_handle="results['weight']"
>>> ifr = IFR.IFR(
    model,
    weights_handle=weights_handle,
    repetition = 50,
    nfolds = 4,
    max_iters = 100,
    cutoff = .6,
    jumpratio = 5,
    max_features_per_iter_ratio = 2,
    verbosity = 2,
    num_gpus_per_worker=0.1
    )

>>> #see feature select method for details
>>> result = x.feature_select(ifr,
            attrname,
            selector_name='IFR',
            f_results_handle='results_',
            append_to_meta=False,
            )

See IFR.fit() to understand the output of IFR.

fit(data, y)¶

Args: data (ndarray of shape (m, n))): array of data, with m the number of observations in R^n. y (ndarray of shape (m))): vector of labels for the data

Return: (pandas.DataFrame) : The dataframe contains the results of IFR. It is indexed by feature_ids and each column contains different information for each feature as described below:

frequency : How many times the feature is extracted

weights : Contains a list of weights, from the weight vectors during training on different partitions. Each value corresponds to the weight for the feature over different extractions. The length of the weights is equal to the frequency.

selection_iteration : Contains a list of indices of the iteration when the feature was extracted over different data partitions. The length of the list is equal to the frequency.

plot_basic_diagnostic_stats(validation_score_iteration_idx=None, n_random_exp=- 1, exit_reason='')¶

select_features_for_data_partition = <ray.remote_function.RemoteFunction object>¶

transform(features, **kwargs)¶

orthrus.sparse.feature_selection.helper module¶

class orthrus.sparse.feature_selection.helper.ReduceIFRFeatures(supervised_attr: str, classifier, scorer, ranking_method_handle, ranking_method_args: dict, parallel: bool = False, verbosity: int = 1, partitioner=None, start: int = 5, end: int = 100, step: int = 5, verbose_frequency: int = 10, num_cpus_per_worker: float = 1.0, num_gpus_per_worker: float = 0.0, local_mode=False)¶: Bases: orthrus.core.pipeline.Transform

orthrus.sparse.feature_selection.helper.get_batch_correction_matric_for_ranked_features(ds, features_dataframe, attr: str, ranking_method_handle, ranking_method_args: dict, batch_correction_metric_handle, batch_correction_metric_args: dict, sample_ids: None, verbose_frequency: int = 10, num_cpus_per_worker: float = 1.0, num_gpus_per_worker: float = 0.0)¶

orthrus.sparse.feature_selection.helper.get_correlates(S, X, c)¶

This function takes a list of feature indices and a data matrix and returns the features from X which have correlation in absolute at least c.

Parameters

S (array-like of shape (n_important_features,) – Feature indices of featues to be correlated to.
X (array-like of shape (n_samples, n_features)) – Data matrix with features.
c (float) – Correlation threshold.

Returns

Correlated features.

Return type

(ndarray)

orthrus.sparse.feature_selection.helper.get_top_95_features(file, attr, cutoff_fraction=0.05)¶

orthrus.sparse.feature_selection.helper.plot_feature_frequency(f_ranks, attr)¶

orthrus.sparse.feature_selection.helper.rank_features_by_attribute(features_df, args)¶

This method takes a features dataframe as input and ranks the features based on a numerical column/attribute.

Parameters

features_df (pandas.DataFrame) – This is a features dataframe that contains result of a feature selection. (check orthrus.core.dataset.DataSet.feature_select method for details)
args (dict) –
This dictionary contains variables to determine which attribute to rank feature on and the order of ranking. Check details for various key and values below: ‘attr’ (Mandatory): Attribute/column name from features_df to rank the features on ‘order’: Whether to rank in ascending or descending order. ‘asc’ for ascending and ‘desc’ for descending.

(defaul: ‘desc’)

’feature_ids’ (list-like): List of identifiers for the features to limit the ranking within certain features only.
e.g. [True, False, True], [‘gene1’, ‘gene3’], etc…, can also be pandas series or numpy array. Default: None, which corresponds to using all features.

Returns

n = number of features in feature_ids or features_df (if feature_ids is None). It contains sorted feature ids (index of features_df).

Return type

ndarray of shape (n, ))

Examples

>>> import orthrus.core.dataset as DS
>>> import orthrus.sparse.feature_selection.IterativeFeatureRemoval as IFR

>>> x = DS.load_dataset(file_path)
>>> ifr = IFR.IFR(
    verbosity = 2,
    nfolds = 4,
    repetition = 500,
    cutoff = .6,
    jumpratio = 5,
    max_iters = 100,
    max_features_per_iter_ratio = 2
    )
>>> result = x.feature_select(ifr,
            attrname,
            selector_name='IFR',
            f_results_handle='results',
            append_to_meta=False,
            )
>>> features_df = results['f_results']
>>> feature_subset = features_df['frequency'] > 5
>>> ranking_method_args = {'attr': 'frequency', feature_ids: feature_subset}

feature will ranked on frequency attribute in descending and will occur only within feature_subset features. >>> ranked_feature_ids = rank_features_by_attribute(features_df, ranking_method_args)

orthrus.sparse.feature_selection.helper.rank_features_by_mean_attribute_value(features_df, args)¶

This method takes a features dataframe as input and ranks the features based on the mean/meadian value a column/attribute, which contains list of numerical data.

Parameters

features_df (pandas.DataFrame) – This is a features dataframe that contains result of a feature selection. (check orthrus.core.dataset.DataSet.feature_select method for details)
args (dict) –
This dictionary contains variables to determine which attribute to rank feature on and the order of ranking. Check details for various key and values below: ‘attr’ (Mandatory): Attribute/column name from features_df to rank the features on ‘order’: Whether to rank in ascending or descending order. ‘asc’ for ascending and ‘desc’ for descending.

(defaul: ‘desc’)

’feature_ids’ (list-like): List of identifiers for the features to limit the ranking within certain features only.
e.g. [True, False, True], [‘gene1’, ‘gene3’], etc…, can also be pandas series or numpy array. Default: None, which corresponds to using all features.

’method’: which operation to perform, can be “mean” or “median”. (Default: “mean”)

Returns

n = number of features in feature_ids or features_df (if feature_ids is None). It contains sorted feature ids (index of features_df).

Return type

ndarray of shape (n, ))

Examples

>>> import orthrus.core.dataset as DS
>>> import orthrus.sparse.feature_selection.IterativeFeatureRemoval as IFR

>>> x = DS.load_dataset(file_path)
>>> ifr = IFR.IFR(
    verbosity = 2,
    nfolds = 4,
    repetition = 500,
    cutoff = .6,
    jumpratio = 5,
    max_iters = 100,
    max_features_per_iter_ratio = 2
    )
>>> result = x.feature_select(ifr,
            attrname,
            selector_name='IFR',
            f_results_handle='results',
            append_to_meta=False,
            )
>>> features_df = results['f_results']
>>> ranking_method_args = {'attr': 'selection_iteration', 'order': asc, 'method': 'median'}
The selection_iteration column in the features_df contains a list of integer values.
In this example the features will be ranked by the median value of selection_iteration in ascending order.
>>> ranked_feature_ids =  rank_features_by_mean_attribute_value(features_df, ranking_method_args)

orthrus.sparse.feature_selection.helper.rank_features_within_attribute_class(features_df, feature_class_attribute, new_feature_attribute_name, x, partitioner, sample_ids, scorer, classification_attr, classifier_factory_method, f_weights_handle, feature_ids=None, **kwargs)¶

orthrus.sparse.feature_selection.helper.reduce_feature_set_size(ds, features_dataframe, sample_ids, attr: str, classifier, scorer, ranking_method_handle, ranking_method_args: dict, partitioner=None, test_sample_ids=None, start: int = 5, end: int = 100, step: int = 5, verbose_frequency: int = 10, num_cpus_per_worker: float = 1.0, num_gpus_per_worker: float = 0.0, local_mode=False, **kwargs)¶

This method takes a features dataframe (output of a feature selection), ranks them by a ranking method and performs a feature set reduction using grid search method, which is defined by start, end and jump parameters.

Different training and test data may be used and the results will change accordingly. The following choices are available:

if test_sample_ids is None and partitioner is None:

Only training data is available, so the results will contain score on the Training data

if test_sample_ids is not None and partitioner is None:

Model is trained on all sample_ids, and then it is then evaluated on test_sample_ids. Results contain evaluation score on test_sample_ids.

if test_sample_ids is None and partitioner is not None:

Model is trained on partitions of sample_ids created by partitioner; results will contain the mean validation score, obtained during validation on these different partitions.

if test_sample_ids is not None and partitioner is not None:

Model is trained using partitions of sample_ids defined by partitioner, and then the test_sample_ids are evaluated using all models. Results contain mean evaluation score on test_sample_ids.

Parameters

features_df (pandas.DataFrame) – This is a features dataframe that contains result of a feature selection. (check orthrus.core.dataset.DataSet.feature_select method for details)
sample_ids (like-like) – List of indicators for the samples to use for training. e.g. [1,3], [True, False, True], [‘human1’, ‘human3’], etc…, can also be pandas series or numpy array.
attr (string) – Name of metadata attribute to classify on.
classifier (object) – Classifier to run the classification experiment with; must have the sklearn equivalent of a fit and predict method.
scorer (object) – Function which scores the prediction labels on training and test partitions. This function should accept two arguments: truth labels and prediction labels. This function should output a score between 0 and 1 which can be thought of as an accuracy measure. See sklearn.metrics.balanced_accuracy_score for an example.
ranking_method_handle (method handle) – handle of the feature ranking method
ranking_method_args (dict) – argument dictionary for the feature ranking method
partitioner (object) – Class-instance which partitions samples in batches of training and test split. This instance must have the sklearn equivalent of a split method. The split method returns a list of train-test partitions; one for each fold in the experiment. See sklearn.model_selection.KFold for an example partitioner. (default = None, check the method description above to see how this affects the results)
test_sample_ids – List of indicators for the samples to use for testing. e.g. [1,3], [True, False, True], [‘human1’, ‘human3’], etc…, can also be pandas series or numpy array. (default = None, check the method description above to see how this affects the results)
start (int) – starting point of the grid search. (default: 5)
end (int) – end point of the grid search. Use -1 to set end as the size of features (default: 100)
step (int) – gap between each sampled point in the grid (default: 5)
verbose_frequency (int) – this parameter controls the frequency of progress outputs for the ray workers to console; an output is printed to console after every verbose_frequency number of processes complete execution. (default: 10)
num_cpus_per_worker (float) –
Number of CPUs each worker needs. This can be a fraction, check ray specifying required resources for more details. (default: 1.)
num_gpus_per_worker (float) –
Number of GPUs each worker needs. This can be fraction, check ray specifying required resources for more details. (default: 0.)

Returns

contains 2 key-value pairs:

’optimal_n_results’: ndarray of shape (m, 2), with m being the total values sampled from the grid in the search. The first column contains the number of top features (different values sampled from the grid search), and the second column contains the score. The array is sorted by score in descending order.

’reduced_feature_ids’ : ndarray of shape (n, ), n is the smallest number of features, out of the m sampled values, that produced the highest score. It contains reduced features ids (index of features_df).

Return type

(dict)

Example

>>> import orthrus.core.dataset as DS
>>> import orthrus.sparse.feature_selection.IterativeFeatureRemoval as IFR

>>> x = DS.load_dataset(file_path)
>>> ifr = IFR.IFR(
...         verbosity = 2,
...         nfolds = 4,
...         repetition = 500,
...         cutoff = .6,
...         jumpratio = 5,
...         max_iters = 100,
...         max_features_per_iter_ratio = 2
...         )
>>> result = x.feature_select(ifr,
            attrname,
            selector_name='IFR',
            f_results_handle='results',
            append_to_meta=False,
            )
>>> features_df = results['f_results']

>>> classifier =  svm.LinearSVC(dual=False)

>>> bsr = sklearn.metrics.balanced_accuracy_score

>>> ranking_method_args = {'attr': 'frequency'}

>>> partitioner = KFold(n_splits=5, shuffle=True, random_state=0)

>>> import orthrus.sparse.feature_selection.helper as fhelper

>>> reduced_feature_results = fhelper.reduce_feature_set_size(x,
                        features_df,
                        sample_ids_training,
                        attrname,
                        classifier,
                        bsr,
                        fhelper.rank_features_by_attribute,
                        ranking_method_args,
                        patitioner=partitioner,
                        start = 5,
                        end = 100,
                        step = 1,
                        verbose_frequency=10,
                        num_cpus_per_worker=2.)

>>> print(reduced_feature_results)

orthrus.sparse.feature_selection.helper.sliding_window_classification_on_ranked_features(ds, features_dataframe, sample_ids, attr: str, model, scorer, ranking_method_handle, ranking_method_args: dict, partitioner=None, test_sample_ids=None, window_size=50, stride=5, verbose_frequency: int = 10, num_cpus_per_worker: float = 1.0, num_gpus_per_worker: float = 0.0, **kwargs)¶

This method takes a features dataframe (output of a feature selection), ranks them by a ranking method and performs classification experiments for various feature sets, which are created by a sliding window approach defined by window size and stride. The result contains score on various feature sets. Different training and test data may be used and the results change accordingly. The following choices are avaible:

if test_sample_ids is None and partitioner is None:

Only training data is available, so the results will contain score on the Training data

if test_sample_ids is not None and partitioner is None:

Model is trained on all sample_ids, and then it is then evaluated on test_sample_ids. Results contain evaluation score on test_sample_ids.

if test_sample_ids is None and partitioner is not None:

Model is trained on partitions of sample_ids created by partitioner; results will contain the mean validation score, obtained during validation on these different partitions.

if test_sample_ids is not None and partitioner is not None:

Model is trained using partitions of sample_ids defined by partitioner, and then the test_sample_ids are evaluated using all models. Results contain mean evaluation score on test_sample_ids.

Parameters

features_df (pandas.DataFrame) – This is a features dataframe that contains result of a feature selection. (check orthrus.core.dataset.DataSet.feature_select method for details)
sample_ids (like-like) – List of indicators for the samples to use for training. e.g. [1,3], [True, False, True], [‘human1’, ‘human3’], etc…, can also be pandas series or numpy array.
attr (string) – Name of metadata attribute to classify on.
classifier (object) – Classifier to run the classification experiment with; must have the sklearn equivalent of a fit and predict method.
scorer (object) – Function which scores the prediction labels on training and test partitions. This function should accept two arguments: truth labels and prediction labels. This function should output a score between 0 and 1 which can be thought of as an accuracy measure. See sklearn.metrics.balanced_accuracy_score for an example.
ranking_method_handle (method handle) – handle of the feature ranking method
ranking_method_args (dict) – argument dictionary for the feature ranking method
partitioner (object) – Class-instance which partitions samples in batches of training and test split. This instance must have the sklearn equivalent of a split method. The split method returns a list of train-test partitions; one for each fold in the experiment. See sklearn.model_selection.KFold for an example partitioner. (default = None, check the method description above to see how this affects the results)
test_sample_ids – List of indicators for the samples to use for testing. e.g. [1,3], [True, False, True], [‘human1’, ‘human3’], etc…, can also be pandas series or numpy array. (default = None, check the method description above to see how this affects the results)
window_size (int) – The number of features to contain in each window. Default is 50.
stride (int) – Controls the stride of the windows. Default is 1.
verbose_frequency (int) – this parameter controls the frequency of progress outputs for the ray workers to console; an output is printed to console after every verbose_frequency number of processes complete execution. (default: 10)
num_cpus_per_worker (float) –
Number of CPUs each worker needs. This can be a fraction, check ray specifying required resources for more details. (default: 1.)
num_gpus_per_worker (float) –
Number of GPUs each worker needs. This can be fraction, check ray specifying required resources for more details. (default: 0.)

Returns

The first column contains the starting position of the window, and the second column contains the score. The array is sorted by first column in ascending order.

Return type

(ndarray of shape (num_windows, 2))

Example

>>> import orthrus.core.dataset as DS
>>> import orthrus.sparse.feature_selection.IterativeFeatureRemoval as IFR

>>> x = DS.load_dataset(file_path)
>>> ifr = IFR.IFR(
    verbosity = 2,
    nfolds = 4,
    repetition = 500,
    cutoff = .6,
    jumpratio = 5,
    max_iters = 100,
    max_features_per_iter_ratio = 2
    )
>>> result = x.feature_select(ifr,
            attrname,
            selector_name='IFR',
            f_results_handle='results',
            append_to_meta=False,
            )
>>> features_df = results['f_results']

>>> classifier = svm.LinearSVC(dual=False)

>>> bsr = sklearn.metrics.balanced_accuracy_score

>>> ranking_method_args = {'attr': 'frequency'}

>>> partitioner = KFold(n_splits=5, shuffle=True, random_state=0)

>>> import orthrus.sparse.feature_selection.helper as fhelper

>>> sliding_window_results = fhelper.sliding_window_classification_on_ranked_features(x,
                        features_df,
                        sample_ids_training,
                        attrname,
                        classifier,
                        bsr,
                        fhelper.rank_features_by_attribute,
                        ranking_method_args,
                        patitioner=partitioner,
                        window_size = 50,
                        stride = 5,
                        verbose_limit=10,
                        num_cpus_per_worker=2.0)

>>> print(sliding_window_results)

orthrus.sparse.feature_selection.kffs module¶

class orthrus.sparse.feature_selection.kffs.KFFS(k=5, n=10, classifier=None, f_weights_handle=None, f_rnk_func=None, training_ids=None, random_state=0)¶

Bases: sklearn.base.BaseEstimator

K-fold Feature Selection (kFFS) selects features using a classifier in a k-fold cross-validation experiment. Specifically, the given classifier is trained on each fold, the features in each fold are ranked and sorted, the top n features are selected from each fold, and the features across all folds are collected and ranked by how many times a given feature was in the top n across each fold.

Parameters

k (int) – Indicates the number of folds to use in k-fold partition. Default is 5.
n – (int): Indicates the rank threshold to use for each fold. KFFS grabs the top n features from each fold.
classifier (object) – Class instance of the classifier being used, must contain a fit method.
f_weights_handle (str) – Name of the classifier attribute containing the feature weights for a given fold.
f_rnk_func (object) – Function to be applied to feature weights for feature ranking. Default is None, and the features will be ranked in from least to greatest.
training_ids (list) – Optional list of training ids, to restrict feature selection further.
random_state (int) – Random state to generate k-fold partitions. Default is 0.

classifiers_¶

Contains each classifier trained on each fold.

Type: series

ranks_¶

Contains the final rankings of all the features.

Type: ndarray

results_¶

Contains the feature weights and ranks for each fold, and the final rankings.

Type: ndarray

fit(X, y)¶

Fits the kFFS model to the training data.

Parameters

X (array-like, (n_samples, n_features)) – Training samples to be used for feature selection.
y (array-like, (n_features,)) – Training labels to be used for feature selection.

Returns

inplace method. Results are stored in KFFS.classifiers_, KFFS.ranks_, and KFFS.results_.

transform(X, n_top_features=10)¶

orthrus.sparse.feature_selection.kfold_loso_ifr module¶

This module contains code for implementing an iterative feature removal (IFR) algorithm using leave one subject out (LOSO) partitions.

class orthrus.sparse.feature_selection.kfold_loso_ifr.KFLIFR(classifier: object, weights_handle: str, n_splits_kfold: int, random_state_kfold=None, train_test_splits=None, gamma: float = 0.01, n_top_features: int = None, jump_ratio: float = None, sort_freq_classes=False, imputer=None)¶

Bases: sklearn.base.BaseEstimator

This iterative feature removal (IFR) algorithm produces ranked feature sets from a single classifier and a data set.

Parameters

classifier (object) – The classifier used to select features. The classifier should produce feature weights and ideally be sparse, so that relatively few features are weighted heavily compared to the total.
weights_handle (str) – The name of the KFLIFR.classifier attribute where the weights are stored. The weights stored there should be an ndarray.
n_splits_kfold (int) – The number of folds, k used in the k-fold cross validation.
random_state_kfold (int) – The random seed used in generating the k-fold partitions. Good for reproducibility of results. The default is None.
train_test_splits (list) – An alternate list of (training, test) splits that replace the k-fold cross validation used in the algorithm. This is useful if you have a specific set of splits you want to use. The default is None, but if provided the arguments KFLIFR.n_splits_kfold and KFLIFR.random_state_kfold are ignored.
gamma (float) – The proportion of features used to break the IFR loop. The IFR loop will stop once the number of features extracted reaches this proportion of the total features.
n_top_features (int) – The number of top features to remove at each iteration of IFR. If this parameter is not given then KFLIFR.jump_ratio will be used instead. The default is None.
jump_ratio (float) – The weight ratio used to determine the number of top features to remove for each iteration of IFR. For example let \(r_i = w_{i}/w_{i+1}\) denote the ratio of the \(i\) th largest weight and \(i+1\) largest weight, then KFLIFR.jump_ratio = 2 means that top features will be chosen until \(r_i \geq 2\). If this parameter is not given then the user must provide a constant number of features to prune at each step via the parameter KFLIFR.n_top_features.
sort_freq_classes (bool) – If True the algorithm will sort frequency classes of feature by the mean of normalized weight across a LOSO experiment— providing a unique ranking. The default is False.
imputer (object) – Optional imputer to impute training set values with.

results_¶

Outputs the feature frequencies and rankings for each fold provided by the k-fold partition defined by KFLIFR.n_splits_kfold and KFLIFR.random_state_kfold, or the user defined splits defined by KFLIFR.train_test_splits.

Type: DataFrame

fit(X, y, groups=None)¶

Performs the feature selection algorithm and stores the results in the attribute KFLIFR.results_.

Parameters

X (array-like of shape (n_samples, n_features)) – The data array used to select features via KFLIFR.classifier.
y (array-like of shape (n_samples, )) – The labels used to select features via KFLIFR.classifier.
groups (array-like of shape (n_samples, )) – Optional set up of labels defining the groups used in a Leave-One-Group-Out experiment. This is useful if you don’t want members of the same group in both the training in test for a LOSO fold.

Returns

inplace method.

orthrus.sparse.feature_selection.kfold_shuffle_ifr module¶

This module contains code for implementing an iterative feature removal (IFR) algorithm using leave one subject out (LOSO) partitions.

class orthrus.sparse.feature_selection.kfold_shuffle_ifr.KFSIFR(classifier: object, weights_handle: str, n_splits_kfold: int, random_state_kfold=None, train_test_splits=None, gamma: float = 0.6, max_feature_threshold=None, n_splits_shuffle=100, random_state_shuffle=None, train_prop_shuffle=0.8, n_top_features: int = None, jump_ratio: float = None, sort_freq_classes=False, imputer=None)¶

Bases: sklearn.base.BaseEstimator

This iterative feature removal (IFR) algorithm produces ranked feature sets from a single classifier and a data set.

Parameters

classifier (object) – The classifier used to select features. The classifier should produce feature weights and ideally be sparse, so that relatively few features are weighted heavily compared to the total.
weights_handle (str) – The name of the KFSIFR.classifier attribute where the weights are stored. The weights stored there should be an ndarray.
n_splits_kfold (int) – The number of folds, k used in the k-fold cross validation.
random_state_kfold (int) – The random seed used in generating the k-fold partitions. Good for reproducibility of results. The default is None.
train_test_splits (list) – An alternate list of (training, test) splits that replace the k-fold cross validation used in the algorithm. This is useful if you have a specific set of splits you want to use. The default is None, but if provided the arguments KFSIFR.n_splits_kfold and KFSIFR.random_state_kfold are ignored.
gamma (float) – Classification rate used to break the IFR loop. The IFR loop will stop once the number of features extracted reaches this proportion of the total features.
max_feature_threshold (int) – The maximum number of features that may be removed on an iteration of IFR. Default is None.
n_splits_shuffle (int) – The number of shuffle splits to use for the inner loop. Default is 100.
random_state_shuffle (int) – Random seed for shuffle splits.
n_top_features (int) – The number of top features to remove at each iteration of IFR. If this parameter is not given then KFSIFR.jump_ratio will be used instead. The default is None.
jump_ratio (float) – The weight ratio used to determine the number of top features to remove for each iteration of IFR. For example let \(r_i = w_{i}/w_{i+1}\) denote the ratio of the \(i\) th largest weight and \(i+1\) largest weight, then KFSIFR.jump_ratio = 2 means that top features will be chosen until \(r_i \geq 2\). If this parameter is not given then the user must provide a constant number of features to prune at each step via the parameter KFSIFR.n_top_features.
sort_freq_classes (bool) – If True the algorithm will sort frequency classes of feature by the mean of normalized weight across a LOSO experiment— providing a unique ranking. The default is False.
imputer (object) – Optional imputer to impute training set values with.

results_¶

Outputs the feature frequencies and rankings for each fold provided by the k-fold partition defined by KFSIFR.n_splits_kfold and KFSIFR.random_state_kfold, or the user defined splits defined by KFSIFR.train_test_splits.

Type: DataFrame

fit(X, y, groups=None)¶

Performs the feature selection algorithm and stores the results in the attribute KFSIFR.results_.

Parameters

X (array-like of shape (n_samples, n_features)) – The data array used to select features via KFSIFR.classifier.
y (array-like of shape (n_samples, )) – The labels used to select features via KFSIFR.classifier.
groups (array-like of shape (n_samples, )) – Optional set up of labels defining the groups used in a Leave-One-Group-Out experiment. This is useful if you don’t want members of the same group in both the training in test for a LOSO fold.

Returns

inplace method.

transform(X, fold: Union[int, str] = 'mean', n_top_features: Optional[Union[int, float]] = None)¶

Restricts X to the top ranked features. If KFSIFR.sort_freq_classes is False then the transform method will rank off of the frequencies, otherwise it will use the rankings available.

Parameters

X (array-like of shape (n_samples, n_features)) – The data to restrict features on.
fold (int or str) – If an integer then the ranking of the feature will be decided on specfied fold. One can also specify fold = “mean”, indicating to use the mean rank across folds.
n_top_features (int, float, None) – Specifies the number of top features to restrict to, if no value is given the transform method will restrict to the top 1% of the features. If a float between 0 and 1 is provided the n_top_features will be computed by the given proportion of the features.

Returns

Restrict array to the top features.

Return type

(array-like of shape (n_samples, n_top_features))

orthrus.sparse.feature_selection.pathway_selection module¶

Module containing classes for pathway discovery in -omics data.

class orthrus.sparse.feature_selection.pathway_selection.PathwayScore(device: int = - 1, parallel: bool = True, score_type='ratio')¶

Bases: sklearn.base.BaseEstimator

convert_type(x)¶

fit(X, y, pathways)¶

transform(X, y=None)¶

orthrus.sparse.feature_selection.pca_snr_selection module¶

This module contains a class which implements the PCA-SNR feature selection method.

class orthrus.sparse.feature_selection.pca_snr_selection.PCASNR¶

Bases: sklearn.base.BaseEstimator

fit(X, y)¶

plot_snrs()¶

snr(X, y, w)¶

orthrus.sparse.feature_selection.sabs module¶

class orthrus.sparse.feature_selection.sabs.SABS(a=1.0, b=1.0, alpha=0.001, iterations=100, store_updates=False, use_cuda=False, device=None)¶

Bases: sklearn.base.BaseEstimator

fit(X, y)¶