Strings, Unicode and explicit comparison rules

pythonscriptingdata-boundariesindependent-practice

Saved on this device

Full lesson · 4 learning steps Open any exercise for focused practice

Step 1 of 4 · intro card

Strings, Unicode and explicit comparison rules

Concept and contract

Normalize text for the intended comparison while keeping human-language and encoding assumptions visible.

A Python str is Unicode text, while bytes is an encoded byte sequence. The worked function strips outer whitespace and uses casefold for a deliberate caseless comparison key. It does not claim to preserve display capitalization or implement full search relevance.

The distinction is observable with Straße: casefold produces strasse, while lower produces straße. A comparison contract that needs caseless Unicode matching must choose an operation with the appropriate behavior rather than assuming every lowercase operation is equivalent.

Mechanism and worked example

String methods return new strings because str objects are immutable.

String methods return new strings because str objects are immutable. strip removes characters at the ends under its stated default behavior; it does not collapse internal whitespace. casefold targets caseless matching and can change the number of code points. Preserve the original text separately when it is needed for display.

len counts string code points, not a universal count of visible human-perceived characters. Combining marks and emoji sequences make that distinction important. Encoding to UTF-8 is another separate step and changes the representation to bytes rather than normalizing comparison semantics.

Complete worked example

python

def text_key(value):
    return value.strip().casefold()

Reasoning: strip preserves internal whitespace; casefold implements the selected caseless key.

Counterexamples and common mistakes

The counterexample uses lower and fails the declared Straße comparison case.

The counterexample uses lower and fails the declared Straße comparison case. Another mistake lowercases a stored display title permanently and then discovers the original presentation is lost. Normalized keys and user-facing source text can coexist without sharing the same representation.

Do not apply this normalization to arbitrary identifiers or passwords. Their case and whitespace rules may be intentionally significant. This lesson uses a text comparison key only; country codes, role identities and human names require their own explicitly chosen domain rules.

Counterexample: a plausible but incorrect implementation

python

def text_key(value):
    return value.strip().lower()

Reasoning: lower does not produce the same key for Straße.

Independent practice: small Python task

Create a key that strips outer whitespace, casefolds and collapses runs of whitespace into a single ordinary space.

Create a key that strips outer whitespace, casefolds and collapses runs of whitespace into a single ordinary space. Predict data analyst for a title containing mixed tabs and multiple spaces. This task adds one small transformation to the already explained comparison pipeline.

Use split without an explicit separator to identify whitespace-separated pieces, then join them with a single space. That deliberately changes internal whitespace for this title-key contract. Keep the original input string unchanged and explain why this would be inappropriate for some formatted source text.

Step 1: Create the caseless string. Step 2: Split whitespace-separated pieces and join them with one space.

Practice starter: solve before reading the solution

python

def text_key(value):
    # Normalize case and collapse whitespace runs.
    raise NotImplementedError

Reasoning: Add whitespace collapsing without mutating the source string.

Hint 1: split() without a separator handles runs of whitespace.
Hint 2: The join separator should be one ordinary space.

Solution, tests and trade-offs

The solution joins the casefolded whitespace-separated words with one space.

The solution joins the casefolded whitespace-separated words with one space. It handles empty or whitespace-only text by returning an empty key and still converts Straße consistently. The function accepts strings under its stated contract; a general unknown-input parser would need an additional boundary.

Acceptance includes Unicode case behavior, tabs, repeated spaces and an empty result. A single ASCII example such as QA would not expose the difference between lower and casefold or prove that internal whitespace follows the revised practice policy.

Practice solution and acceptance checks

python

def text_key(value):
    return " ".join(value.casefold().split())

Reasoning: Unicode case folding and one-space word joining are independently tested.

Interview reasoning and next step

Explain text, bytes, normalization and display as separate responsibilities in an interview.

Explain text, bytes, normalization and display as separate responsibilities in an interview. A useful answer gives one concrete counterexample to a simplistic character or case assumption. Avoid claiming that casefold is a complete language-sensitive collation or a substitute for an actual search system.

The following condition lesson uses missing values and genuine zero. It applies the same principle: preserve distinctions until the business rule explicitly authorizes a transformation, rather than treating every inconvenient representation as interchangeable.

Optional stretch — after the core task works: Compare a decomposed accented string with a precomposed one and explain why case folding alone is not Unicode normalization.

Computer practice: https://applydjinn.com/typorithm/python-lab-20261105.zip. Run verify.py with your selected Python interpreter, then complete an exercise and check it with --practice followed by the lesson id. The lab uses only the standard library and disposable local fixtures; it never connects to the production database or company feeds.

Normalize text for the intended comparison while keeping human-language and encoding assumptions visible.