Data Science FAQ’s

The Supervised machine learning allows to collect data or produce a data output from the previous experience. It uses labeled datasets to train algorithms that to classify data or predict outcomes accurately.

Unsupervised machine learning is a technique, where you do not need to supervise the model. It helps you to finds all kind of unknown patterns in data.

Selection bias occurs when the sample obtained is not representative of the population intended to be analysed.

There are 4 different types of kernels in SVM.

1. Linear Kernel
2. Polynomial kernel
3. Radial basis kernel
4. Sigmoid kernel

NLP stands for Natural Language Processing. NLP is a branch of artificial intelligence which gives machines the ability to read and understand human languages.

The process of removing sub-nodes of a decision node is called pruning or opposite process of splitting.

Ensemble learning is the art of combining diverse set of learners(Individual models) together to improvise on the stability and predictive power of the model.

Random forest is a versatile machine learning method, which is capable of performing both regression and classification of tasks. It is used for dimentionality reduction, treats missing values, outlier values. It is a type of ensemble learning method, where a group of weak models combine to form a powerful model.

A false positive is an incorrect identification of the presence of a condition when it’s absent.

A false negative is an incorrect identification of the absence of a condition when it’s actually present.

‘Statistical power’ is the power of a binary hypothesis, which is the probability that the test rejects the null hypothesis given that the alternative hypothesis is true.

In the wide-format data, a subject’s repeated responses will be in a single row, and each response is in a separate column.

In the long-format data, each row is a one-time point per subject. We can recognize data in wide format by the fact that columns generally represent groups.

When we perform a hypothesis test in statistics, a p-value can help us to determine the strength of the results. p-value is a number between 0 and 1. Based on the value it will denote the strength of the results.

The mean is the most frequently used measure of central tendency as it uses all the values in the data set to give you an average. For data from the skewed distributions, the median is better than the mean because it is not influenced by extremely large values.

Any type of categorical data will not have a gaussian distribution or lognormal distribution.

Exponential distributions — eg. the amount of time that a car battery lasts or the amount of time until an earthquake occurs.

Resampling is done to  (i) Estimate the accuracy of sample statistics by using subsets of accessible data or drawing randomly with replacement from a set of data points. (ii). Substitute labels on data points when performing significance tests (iii) Validaeg models by using random subsets (bootstrapping, cross-validation).

The Law of Large Numbers is a theory which states that as the number of trials increases, the average of the result will become closer to the expected value.

Eg. flipping heads from fair coin 100,000 times should be closer to 0.5 than 100 times.

To combat overfitting and underfitting, we can resample the data to estimate the model accuracy (k-fold cross-validation) and by having a validation dataset to evaluate the model.

A/B testing is a form of hypothesis testing and two-sample hypothesis testing to compare two versions, the control and variant, of a single variable. It is commonly used to improve and optimize user experience and marketing.

It is the logical error of focusing aspects that support surviving some process and casually overlooking those that did not work because of their lack of prominence. This can lead to wrong conclusions in numerous different means.

In many ways we can control and minimize bias. The 2 common ways include randomization, where participants are assigned by chance, and random sampling, sampling in which each member has an equal probability of being chosen.