Interview Questions — Python, ML & DL

These questions cover the full spectrum of what is asked in software engineering and machine learning interviews — from Python internals to production ML system design. Each question includes the direction a strong answer takes.

Part 1: Python Internals & Engineering

1. What is the GIL and why does it exist? How do you work around it?

Answer: The Global Interpreter Lock (GIL) is a mutex that prevents multiple native threads from executing Python bytecode simultaneously. It exists because CPython's memory management (reference counting) is not thread-safe. The GIL means that pure Python multi-threading cannot achieve true CPU parallelism.

Workarounds:

• Use multiprocessing — each process has its own interpreter and GIL
• Use concurrent.futures.ProcessPoolExecutor for CPU-bound tasks
• Use C extensions that release the GIL (NumPy, TensorFlow do this internally)
• Use asyncio for I/O-bound concurrency (not parallelism)

2. Explain Python's memory model: reference counting, garbage collection, and the cycle collector.

Answer: CPython uses reference counting as the primary mechanism. When an object's reference count drops to zero, it is immediately deallocated. However, reference counting alone cannot handle cyclic references (e.g., object A references B and B references A — both counts stay at 1 forever).

Python's cyclic garbage collector (gc module) periodically scans for these cycles and breaks them. The gc uses three generations — objects that survive more collection cycles are moved to older generations and collected less frequently (generational hypothesis: most objects die young).

3. What is the difference between is, ==, and id()?

Answer:

• == calls __eq__ — compares values
• is compares identity — checks if both names point to the exact same object in memory
• id() returns the memory address of an object

1a = [1, 2, 3]
2b = [1, 2, 3]
3c = a
4
5print(a == b)   # True  — same values
6print(a is b)   # False — different objects in memory
7print(a is c)   # True  — same object, aliased
8print(id(a) == id(c))  # True

Bug trap: is on small integers and interned strings may return True due to caching — do not rely on is for value comparison.

4. When would you use a generator instead of a list? What is the memory difference?

Answer: A list comprehension materializes all elements in memory at once. A generator yields elements lazily — one at a time — using O(1) memory regardless of sequence length.

1# This loads 10 million integers into RAM simultaneously
2big_list = [x**2 for x in range(10_000_000)]
3
4# This uses a constant ~100 bytes regardless of size
5big_gen  = (x**2 for x in range(10_000_000))

Use generators when: processing large files line-by-line, streaming API responses, pipelines where only one item is needed at a time.

5. Explain mutable default arguments — why is this a classic Python bug?

Answer: Default argument values are evaluated once when the function is defined, not on each call. A mutable default (like a list or dict) is shared across all calls.

1# Bug
2def append_item(item, lst=[]):
3    lst.append(item)
4    return lst
5
6print(append_item(1))  # [1]
7print(append_item(2))  # [1, 2] — NOT [2]!
8
9# Fix
10def append_item(item, lst=None):
11    if lst is None:
12        lst = []
13    lst.append(item)
14    return lst

6. What are decorators and how are they implemented?

Answer: A decorator is a function that takes another function as input and returns a modified version. Syntactic sugar for func = decorator(func).

1import functools
2import time
3
4def timer(func):
5    @functools.wraps(func)  # Preserves __name__, __doc__
6    def wrapper(*args, **kwargs):
7        start = time.perf_counter()
8        result = func(*args, **kwargs)
9        elapsed = time.perf_counter() - start
10        print(f"{func.__name__} took {elapsed:.4f}s")
11        return result
12    return wrapper
13
14@timer
15def train_model():
16    time.sleep(0.1)
17
18train_model()

functools.wraps is critical — without it, the wrapped function loses its identity (func.__name__ becomes wrapper).

7. What is __slots__ and when should you use it?

Answer: By default, Python objects store their attributes in a per-instance __dict__ (a hash map). __slots__ declares the allowed attributes at the class level, eliminating the __dict__ entirely.

Benefits: reduces memory per instance by ~30–50%, slightly faster attribute access.

1class Sensor:
2    __slots__ = ["id", "temperature", "active"]
3    def __init__(self, id, temp, active):
4        self.id = id
5        self.temperature = temp
6        self.active = active

Use when creating millions of instances (e.g., ML feature objects, parsed log records).

8. How does Python resolve attribute lookup order (MRO)?

Answer: Python uses C3 Linearization (Method Resolution Order) to determine the order in which base classes are searched when accessing an attribute. ClassName.__mro__ shows the full resolution order.

1class A: pass
2class B(A): pass
3class C(A): pass
4class D(B, C): pass
5
6print(D.__mro__)
7# D → B → C → A → object

Part 2: Data Science & Machine Learning Core

9. What is data leakage? Give three specific ways it occurs.

Answer: Data leakage is when information from the future or from the test set influences the training process, producing models that appear accurate in development but fail in production.

Examples:

1. Scaling before splitting: Fitting StandardScaler on the full dataset before splitting uses test set statistics (mean/std) during training. The model implicitly "knows" the test data distribution.
2. Future features in time-series: Including a feature computed from data at time t+1 to predict at time t.
3. Target encoding with test rows: Computing category-level mean target values using all rows including test — the test set leaks its targets back into training features.

10. Explain precision, recall, and F1. When would you prioritize recall over precision?

Answer:

• Precision = TP / (TP + FP) — of all positive predictions, what fraction were correct?
• Recall = TP / (TP + FN) — of all actual positives, what fraction did we catch?
• F1 = 2 × (P × R) / (P + R) — harmonic mean, penalizes extreme imbalance between P and R

Prioritize recall when: failing to detect a positive is more costly than a false alarm.

• Medical diagnosis: missing a cancer case (FN) is far worse than a false alarm (FP)
• Fraud detection: missing fraud is worse than blocking a legitimate transaction for review
• Safety systems: missing a fault condition in a machine is worse than a spurious alert

Prioritize precision when: false alarms have high cost.

• Spam filter that might delete important emails
• Automated content moderation that might remove legitimate content

11. What is the difference between ROC-AUC and PR-AUC? When is ROC-AUC misleading?

Answer: ROC-AUC measures the probability that a randomly chosen positive sample is ranked higher than a randomly chosen negative sample. It is threshold-independent.

When ROC-AUC is misleading: On heavily imbalanced datasets (e.g., 1% positives, 99% negatives), a model can achieve very high ROC-AUC while having poor precision on the minority class. This happens because ROC-AUC is influenced by the large number of true negatives.

PR-AUC (Precision-Recall AUC) focuses only on the positive class — it is more informative when the class imbalance is severe and positive class performance is what matters.

Rule of thumb: Use ROC-AUC for balanced datasets; use PR-AUC for imbalanced ones.

12. Explain bias-variance tradeoff with a production example.

Answer: Every model's expected error decomposes as: Error ≈ Bias² + Variance + Irreducible Noise

A high-bias model is too simple — a linear regression predicting house prices when the true relationship is non-linear. It consistently underestimates or overestimates across all market conditions.

A high-variance model is too complex — a 100-depth decision tree that memorizes all training house prices. It performs perfectly on training data but fails on any new neighborhood not seen in training.

Production reality: A model with slightly higher training error but much lower train-validation gap (low variance) almost always performs better in production.

13. How do you detect and handle concept drift in production?

Answer: Concept drift means the statistical relationship P(Y|X) changes over time — the same inputs now mean something different.

Detection methods:

• Monitor prediction distribution over time (compare to baseline)
• Track model performance metrics using delayed ground truth labels
• Run statistical tests on input feature distributions (KS-test, PSI score)
• Monitor business KPIs linked to model output

Handling strategies:

• Sliding window retraining on recent data
• Ensemble of models trained on different time windows
• Trigger-based retraining when drift detection fires
• Feature-based drift alerting before performance degrades

14. You have a dataset with 500 features. How do you approach feature selection?

Answer:

1. Correlation filter: Remove features with near-zero variance and features correlated > 0.95 with another feature.
2. Univariate selection: Rank features by mutual information or F-test.
3. Model-based importance: Fit a Random Forest and rank by permutation importance.
4. Recursive Feature Elimination (RFE): Iteratively remove least important features.
5. L1 regularization (Lasso): Let the model zero out irrelevant features automatically.
6. Ablation study: Progressively remove low-importance features and track validation metric change.

Never use test-set performance to drive feature selection. This is a form of leakage.

15. What is calibration and why does it matter separately from AUC?

Answer: A calibrated model produces probabilities that match empirical frequencies. If a model predicts 0.8 probability for 1000 events, approximately 800 of them should actually occur.

A model can have excellent AUC (perfect ranking) but be poorly calibrated (its predicted probabilities are systematically off). This matters whenever downstream decisions use the raw probability — risk scoring, decision thresholds, expected value calculations.

Calibration methods:

• Platt Scaling: fit a logistic regression on top of model scores
• Isotonic Regression: more flexible non-parametric calibration

1from sklearn.calibration import CalibratedClassifierCV, calibration_curve
2import matplotlib.pyplot as plt
3
4cal_model = CalibratedClassifierCV(base_estimator=model, method="isotonic", cv=5)
5cal_model.fit(X_train, y_train)
6
7prob_true, prob_pred = calibration_curve(y_test, cal_model.predict_proba(X_test)[:, 1], n_bins=10)
8plt.plot(prob_pred, prob_true, marker="o")
9plt.plot([0,1],[0,1], linestyle="--")
10plt.xlabel("Mean Predicted Probability")
11plt.ylabel("Fraction of Positives")
12plt.title("Calibration Curve")
13plt.show()

Part 3: Deep Learning & Neural Networks

16. Why do neural networks require non-linear activation functions?

Answer: Without non-linear activations, stacking multiple linear layers is mathematically equivalent to a single linear transformation: W₂(W₁x + b₁) + b₂ = W'x + b'. No matter how many layers you add, the network can only represent a linear function. Non-linear activations (ReLU, tanh, GELU) allow the composition of layers to represent any function (Universal Approximation Theorem).

17. Explain vanishing gradients. Why do LSTMs mitigate this?

Answer: In a deep network or an unrolled RNN, backpropagation multiplies gradients through every layer. If weights are small (< 1), gradients shrink exponentially as they flow backward. Earlier layers receive almost no signal and learn very slowly or not at all.

LSTMs mitigate this through their cell state c_t. The cell state is updated via additive operations controlled by gates — the gradient can flow backward through time without being multiplied by a chain of weight matrices. This gives LSTMs a near-constant error carousel for long sequences.

18. What is Batch Normalization and why does it speed up training?

Answer: Batch Normalization normalizes the activations of a layer across the current mini-batch to have zero mean and unit variance, then applies learnable scale γ and shift β parameters.

x_norm = (x - μ_batch) / √(σ²_batch + ε) y = γ x_norm + β

Why it helps:

• Reduces internal covariate shift — layer inputs remain in a stable range regardless of upstream parameter changes
• Allows much higher learning rates (reduces sensitivity to initialization)
• Acts as mild regularization (batch statistics add noise)

Caution: Batch norm behavior differs between training and inference (uses running statistics at inference). Applying it to very small batches or recurrent steps is problematic — use Layer Norm instead for those cases.

19. Compare Dropout, L2 regularization, and Early Stopping.

Answer:

They are complementary and can be combined. Early stopping is free — always use it.

20. How does the Adam optimizer work? What are its advantages over SGD?

Answer: Adam combines two ideas:

• Momentum (1st moment): maintains a moving average of gradients — smooths out noisy updates
• RMSProp (2nd moment): maintains a moving average of squared gradients — adapts the learning rate per parameter

Update rule: m_t = β₁m_{t-1} + (1-β₁)g_t v_t = β₂v_{t-1} + (1-β₂)g_t² w_{t+1} = w_t - η · m̂_t / (√v̂_t + ε)

Advantages over SGD:

• Works well with sparse gradients
• Adapts learning rate per parameter — no single global lr needed
• Converges faster on most architectures

Disadvantage: Can generalize slightly worse than well-tuned SGD+momentum on some tasks. Many practitioners use AdamW (Adam with decoupled weight decay) for final production training.

21. What is transfer learning? When can it fail?

Answer: Transfer learning initializes a model with weights pretrained on a large dataset (ImageNet, large text corpora) before fine-tuning on your smaller target dataset. It works because early layers learn generic, reusable features (edges, textures, syntax patterns) that transfer across domains.

When it works: Your target domain is similar to the pretraining domain (e.g., medical images → natural images are somewhat transferable).

When it fails:

• Domain mismatch is severe (e.g., satellite imagery → pretraining on Instagram photos)
• Catastrophic forgetting: aggressive fine-tuning overwrites pretrained knowledge. Fix: freeze early layers, train only later layers first, then gradually unfreeze.
• Pretraining data distribution is biased in ways that harm your use case.

22. Explain the attention mechanism in one paragraph.

Answer: Given a query vector q and a set of key-value pairs (K, V), attention computes a weighted sum of the values where the weights are determined by how similar each key is to the query. The similarity is computed as a scaled dot product qKᵀ / √d_k, passed through softmax to produce a probability distribution over positions, and applied to V. This allows the model to directly route information from any position to any other — regardless of sequence length — enabling it to capture long-range dependencies that RNNs struggle with.

23. What causes exploding gradients and how do you fix it?

Answer: Exploding gradients occur when weight matrices have eigenvalues greater than 1 — gradient magnitudes grow exponentially during backpropagation. Symptoms: loss suddenly becomes NaN, or grows wildly after initially decreasing.

Fixes:

• Gradient Clipping: most common fix — clips gradient norm to a maximum value: optimizer = Adam(clipnorm=1.0)
• Better weight initialization (Xavier, He init)
• Batch Normalization to stabilize activations
• Reduce learning rate

Part 4: MLOps & System Design

24. How do you design a monitoring system after a model ships to production?

Answer: Monitoring must cover both the input data pipeline and the model's output behavior:

Input monitoring:

• Feature distribution drift (PSI score, KS-test per feature)
• Missing value rate changes
• Data freshness — is the pipeline delivering data on time?

Output monitoring:

• Prediction distribution (mean, std, percentiles) vs. baseline
• Calibration drift
• Segment-wise performance (some segments degrade before others)

Delayed label monitoring:

• When ground truth is available (e.g., transaction was fraud or not), compare against model prediction
• Track F1/recall on rolling windows

Infrastructure:

• Latency and error rate of the inference endpoint
• Throughput under load

25. How do you run a safe model rollout?

Answer: Never replace a production model instantaneously.

1. Shadow mode: new model processes all requests but its predictions are logged, not served. Compare offline metrics without business risk.
2. Canary release: route 1–5% of traffic to the new model. Monitor for anomalies in predictions and infrastructure metrics.
3. Guardrail metrics: define hard thresholds (e.g., if conversion rate drops > 2%, auto-rollback).
4. Gradual rollout: increase traffic percentage incrementally (5% → 20% → 50% → 100%) with hold periods between steps.
5. Rollback policy: define exactly what triggers a rollback and who has authority to execute it.

26. A data scientist hands you a Jupyter notebook and says "put this in production." What is your process?

Answer:

1. Code review for correctness: verify there is no leakage, random seeds are set, scaling is inside a Pipeline.
2. Refactor: move preprocessing into reproducible Pipeline; extract feature engineering into versioned modules.
3. Testing: write unit tests for preprocessing functions; integration tests for the full inference path with sample inputs.
4. Dependency pinning: freeze library versions into requirements.txt or a Docker image.
5. Containerize: package the model, dependencies, and inference code into a Docker container.
6. API layer: wrap inference in a FastAPI or Flask service with input schema validation.
7. Monitoring hooks: add logging for input features and output predictions to enable drift monitoring.
8. CI/CD pipeline: model retraining and deployment should be automated and triggered by code/data changes.

Part 5: Behavioral-Technical Bridge

27. Describe a model you rejected despite good metrics.

Strong direction: Discuss a case where the metric looked good but the model was making predictions based on spurious correlations, had poor behavior on important subgroups, or would be impossible to audit in a regulated environment. Show that you evaluated the model beyond a single number.

28. Tell me how you would explain a model failure to a non-technical stakeholder.

Strong direction: Translate model failure into business impact language. Avoid technical jargon. Explain what the model got wrong, what data conditions caused it, what has been done to fix it, and what the monitoring plan is going forward to catch similar issues earlier.

29. How do you decide when a model is "good enough" to ship?

Strong direction: This is never a metric threshold alone. Consider: does the model outperform the current system (A/B test)? Is it within latency and memory budgets? Is there a rollback plan? Has it been validated on a held-out dataset that reflects the deployment population? Has it been reviewed for bias on critical subgroups?

Quick Reference: Things Interviewers Love to Hear