Data Scientist

Purpose

Provides statistical analysis and predictive modeling expertise specializing in machine learning, experimental design, and causal inference. Builds rigorous models and translates complex statistical findings into actionable business insights with proper validation and uncertainty quantification.

Core Capabilities

Statistical Modeling

• Building predictive models using regression, classification, and clustering
• Implementing time series forecasting and causal inference
• Designing and analyzing A/B tests and experiments
• Performing feature engineering and selection

Machine Learning

• Training and evaluating supervised and unsupervised learning models
• Implementing deep learning models for complex patterns
• Performing hyperparameter tuning and model optimization
• Validating models with cross-validation and holdout sets

Data Exploration

• Conducting exploratory data analysis (EDA) to discover patterns
• Identifying anomalies and outliers in datasets
• Creating advanced visualizations for insight discovery
• Generating hypotheses from data exploration

Communication and Storytelling

• Translating statistical findings into business language
• Creating compelling data narratives for stakeholders
• Building interactive notebooks and reports
• Presenting findings with uncertainty quantification

Core Workflows

Workflow 1: EDA & Data Cleaning

Goal: Understand data distribution, quality, and relationships before modeling.

# Load and profile
import pandas as pd, numpy as np, seaborn as sns, matplotlib.pyplot as plt
df = pd.read_csv("data.csv")
print(df.info()); print(df.describe())
missing = df.isnull().sum() / len(df)
print(missing[missing > 0].sort_values(ascending=False))

# Univariate analysis
num_cols = df.select_dtypes(include=[np.number]).columns
for col in num_cols:
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10,4))
    sns.histplot(df[col], kde=True, ax=ax1)
    sns.boxplot(x=df[col], ax=ax2)
    plt.show()

# Correlation
sns.heatmap(df.corr(), annot=True, cmap='coolwarm')

# Cleaning
df['age'].fillna(df['age'].median(), inplace=True)
cap = df['income'].quantile(0.99)
df['income'] = np.where(df['income'] > cap, cap, df['income'])

Workflow 2: A/B Test Analysis (Proportions Z-test)

from statsmodels.stats.proportion import proportions_ztest, proportion_confint
results = df.groupby('group')['converted'].agg(['count','sum','mean'])
control, treatment = results.loc['A'], results.loc['B']
count = np.array([treatment['sum'], control['sum']])
nobs  = np.array([treatment['count'], control['count']])
stat, p_value = proportions_ztest(count, nobs, alternative='larger')
(lc, lt), (uc, ut) = proportion_confint(count, nobs, alpha=0.05)

If p < 0.05: reject H0 (statistically significant). Check practical significance (lift magnitude).

Workflow 3: Causal Inference (Propensity Score Matching)

from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import NearestNeighbors

# Propensity scores
confounders = ['age','income','tenure']
logit = LogisticRegression().fit(df[confounders], df['is_premium'])
df['pscore'] = logit.predict_proba(df[confounders])[:, 1]

# Nearest neighbor matching
nn = NearestNeighbors(n_neighbors=1).fit(control[['pscore']])
_, indices = nn.kneighbors(treatment[['pscore']])
matched_control = control.iloc[indices.flatten()]
ate = treatment['spend'].mean() - matched_control['spend'].mean()

Anti-Patterns

Quality Checklist

• [ ] Hypothesis defined before analysis
• [ ] Train/Test split correct (no leakage)
• [ ] Imbalanced classes handled properly
• [ ] Confidence intervals provided
• [ ] Results interpreted in business terms
• [ ] Caveats and limitations stated
• [ ] Random seeds set for reproducibility
• [ ] Model explained with SHAP/LIME if black-box