AI Tip from @karpathy

Description

Automatically generated AI learning skill from curated web and social media sources.

Steps

1. Tip: When training neural nets, always start with a tiny dataset (e.g. 10 examples) and overfit it completely before scaling up. This validates your entire pipeline works end-to-end.
2. # Test your training loop
3. def test_overfit():
4. model = YourModel()
5. tiny_dataset = get_tiny_dataset(10)
6. train(model, tiny_dataset, epochs=1000)
7. assert loss < 0.01

Code Examples

# Test your training loop
def test_overfit():
    model = YourModel()
    tiny_dataset = get_tiny_dataset(10)
    train(model, tiny_dataset, epochs=1000)
    assert loss < 0.01