Section 1.5: Strings, Files, and Data Formats

Data arrives as text and leaves as text. Between those two points it has to be read from disk without leaking a file handle, split into fields, stripped of the whitespace and quoting that came with it, converted into real types, and written back in a form something else can read. This section covers the string operations that do the splitting and cleaning, the safe way to open and close a file, and the two structured formats you will meet constantly, comma-separated values and JSON.

Section 4 gave you the structures to hold a table in memory. Almost every one of those structures was built by typing the data in by hand. That is the unfinished half: a table whose rows are a literal written into the source file is a demonstration, not a tool. This section connects those structures to files on disk, which is where real data actually is.

Lessons

  1. Working With Strings - normalizing, splitting, joining, stripping and formatting text.
  2. Reading and Writing Files Safely - opening, reading, writing, and closing without leaking a handle.
  3. Parsing Structured Text - CSV and JSON - turning a delimited text format into Python objects, and a nested one into Python objects and back.

By the end you will be able to:

  • Transform raw text into cleaned fields with string operations, and cast each field to its real type.
  • Read and write a file in a way that closes it even when an error occurs partway.
  • Parse a delimited file by hand and explain what a quoted field containing a delimiter does to a naive split.
  • Round-trip a nested structure to JSON and back, and prove the result is unchanged.