Path 66 · The clean-room notebook · Intermediate
Keep the evidence when you fix the data.
A guided, practical route through python data cleaning.
What you will be able to do
Build a repeatable Python cleaner that normalizes fields, rejects invalid rows and produces an audit summary without overwriting its input.
Plan about 10h 45m in total. Reading and practice are editorial estimates; video runtimes come from public provider metadata. Your pace will vary.
Before you begin
Check your starting point
What you need
Basic Python variables, loops and functions. Python 3 and its standard library are sufficient.
The selected public provider lessons are free to view. The original case can be completed with paper, a text editor or freely available local tools. Paid exams, certificates, subscriptions and optional provider software are not required for this project.
Completing this WikiFree path does not award a provider certificate. Any provider assessment, account requirement or credential must be checked separately on its official page.
Three readiness questions
Should a cleaner silently fill unknown names?
No. Preserve the missingness and apply the stated rule.
What makes a rerun reproducible?
The same input and rules produce the same outputs.
Why keep rejected rows?
They explain the difference between input and accepted output.
Use a workable pace
At 5 hours per week, allow about 3 weeks. Start with Stage 1, continue in order and reserve 2h 30m for the capstone. If a gate fails, use its repair step before advancing. At another weekly budget, divide 645 estimated minutes by your available weekly minutes and round up.
Keep a small evidence folder. A checked box means you reviewed your output; it is not a certificate or proof that a provider lesson was watched.
The working case
Five rows, an explained outcome
A fictional workshop export contains five registration rows. IDs should be unique, names nonblank and seats integers from 1 to 4. Keep the first valid occurrence of an ID; subsequent duplicates are rejected. Whitespace is normalized, but missing facts are never guessed.
| id | name | seats |
|---|---|---|
| R1 | Ada | 2 |
| R2 | Ben | 0 |
| R1 | Ada | 2 |
| R3 | Chen | 3 |
| R4 | 1 |
The table and explanations preserve the essential case information when the diagram is unavailable.
Your route
Five outputs, then a final handoff
- 01Profile before changingData contract and profile
About 2h 23m including assigned learning and practice. - 02Normalize without inventingNormalization function
About 1h 15m including assigned learning and practice. - 03Apply validation and duplicate rulesCleaner with accepted and rejected outputs
About 2h 18m including assigned learning and practice. - 04Test edge casesTest evidence and new case
About 1h 20m including assigned learning and practice. - 05Package an auditable runReproducible cleaning run
About 1h including assigned learning and practice. - FINALCapstone reviewCombine the evidence into one inspectable handoff.
About 2h 30m.
0 of 6 outputs checked. JavaScript adds browser-only progress saving.
The complete lesson and all downloads remain available. Use the printable project workbook to track your progress.
Use the minimum useful source material
The courses behind this route
Learn Skills Daily
Data Analysis with Python: Free Course
The existing Microsoft Python-for-data playlist supports Python data-handling foundations; the cleaner and checks are original standard-library exercises.
Selected material only; this path does not claim completion of the entire provider course.
Course publication and selected public lesson metadata were checked on 13 October 2026. This is not a claim of full video playback, provider-course completion or permanent availability. Recheck the linked source if a lesson changes.
Learn, make, inspect
The five-stage workshop
Profile before changing
Video 1h 12m · reading budget 0m · original practice about 1h 10m.
Inspect row counts, field names and value patterns before cleaning. A plausible value can still violate the case rule. Record the raw text and identify which transformations are allowed. This path uses the Python standard library for transparent rules; it does not require a paid data platform or an installed notebook service.
Watch these selected lessons
Do the work
Profile the five rows and write the three acceptance rules.
Save this output
Data contract and profile
Pass the review gate
The profile identifies the duplicate, invalid seat count and blank name.
If you need another attempt
Inspect every field as text before converting it.
Normalize without inventing
Video 0m · reading budget 0m · original practice about 1h 15m.
Normalization changes representation while preserving meaning. Strip surrounding whitespace and parse seats carefully; do not turn an invalid string into zero. Keep the raw row alongside an error reason. Functions with explicit inputs and outputs are easier to inspect than edits performed manually in a spreadsheet.
Original application stage. Use your preceding outputs and the supplied case; there is no additional required provider lesson.
Do the work
Implement name trimming and safe integer parsing. Keep the original input file unchanged.
Save this output
Normalization function
Pass the review gate
Ada loses surrounding spaces; no missing name is fabricated.
If you need another attempt
Return an error reason when parsing fails instead of a default value.
Apply validation and duplicate rules
Video 48m · reading budget 0m · original practice about 1h 30m.
Validation and deduplication order affects the result. This case accepts the first valid occurrence of an ID, then rejects later occurrences. Document that choice and test it. A rejected invalid row should not reserve an ID forever unless the data contract explicitly says so.
Watch these selected lessons
Do the work
Implement row validation, accepted-ID tracking and rejection reasons.
Save this output
Cleaner with accepted and rejected outputs
Pass the review gate
Accepted IDs are R1 and R3 with total seats 5; three rows are rejected.
If you need another attempt
Track IDs only after a row passes the field rules.
Test edge cases
Video 0m · reading budget 0m · original practice about 1h 20m.
Useful tests challenge the rules rather than copy the code. Include a non-integer seat value, boundary values 1 and 4, a blank ID and an invalid first occurrence followed by a valid row with the same ID. Compare outputs and confirm the input file is byte-for-byte unchanged.
Original application stage. Use your preceding outputs and the supplied case; there is no additional required provider lesson.
Do the work
Run the supplied reference checks, then add one new edge case.
Save this output
Test evidence and new case
Pass the review gate
The rule boundaries and duplicate ordering are both exercised.
If you need another attempt
Name the expected behavior before running the test.
Package an auditable run
Video 0m · reading budget 0m · original practice about 1h .
An audit summary reconciles the number of input, accepted and rejected rows and records the rules used. Avoid calling a file clean without describing its scope. A successful run proves conformity to these checks, not correctness of every real-world fact in the data.
Original application stage. Use your preceding outputs and the supplied case; there is no additional required provider lesson.
Do the work
Produce accepted.csv, rejected.csv and summary.json in a new output folder.
Save this output
Reproducible cleaning run
Pass the review gate
Input count equals accepted plus rejected and the total accepted seats is 5.
If you need another attempt
Add a reconciliation assertion before writing a success message.
Make the result inspectable
Your final project
Deliver the data contract, cleaner, input-preservation evidence, outputs, tests and a short limitations note. Add a new malformed seat value and demonstrate that the run continues with a recorded rejection.
The ZIP includes synthetic input data, a project workbook, worked reference, evidence log, scoring rubric and case-specific materials. Code projects include source and clearly stated local execution requirements.
Use one consistent rubric
| Criterion | Maximum | Evidence |
|---|---|---|
| Evidence and correctness | 30 | Calculations, source use and results match the supplied case; limitations are honest. |
| Required artifacts | 30 | All five stage outputs and the capstone handoff are present and inspectable. |
| Reasoning and decisions | 20 | Choices, assumptions, alternatives and the case-specific critical condition are explained. |
| Reproducibility | 10 | A reviewer can follow the stated steps and distinguish expected from observed results. |
| Clear communication | 10 | Labels, units, structure and evidence references make the work easy to review. |
Score at least 80/100, supply every required artifact, and satisfy the path-specific critical condition. Rework missing evidence before marking complete.
The critical condition
No input file is overwritten and every excluded row has an explicit reason.
Compare with a worked reference
The accepted rows are R1/Ada/2 and R3/Chen/3. R2 fails the seat rule, the later R1 is a duplicate, and R4 lacks a name. Thus 5 input rows = 2 accepted + 3 rejected, with 5 accepted seats. The supplied reference uses only the Python standard library.
Know the boundary
The tiny case teaches auditable cleaning, not a full production data pipeline or verification of personal information.
Questions before you finish
Do I need to finish every source course?
No. This path names selected lessons or reading sections. Complete the full provider course only if it supports your wider goal. The path does not claim equivalence to that course.
How do I know when to advance?
Use the output and review gate in each stage. If evidence is missing or a result conflicts with the case, follow the repair step and retry before advancing.
Can I do the project without paid software?
The selected public provider lessons are free to view. The original case can be completed with paper, a text editor or freely available local tools. Paid exams, certificates, subscriptions and optional provider software are not required for this project.
What does my checklist prove?
It records your own review in this browser only. It does not verify viewing, skill, a provider credential or a real-world deployment.
What if a lesson or tool changes?
Use the linked course and provider page to inspect current availability. Record the change and revisit the relevant assignment; do not invent a replacement lesson or claim an unavailable activity was completed.
Questions & discussion
Share a useful question or correction. Comments appear after moderation. Please avoid personal or sensitive information.