Missing values, nullable dtypes and three-valued logic

A missing value represents unavailable information, not an observed zero. Pandas offers several missing sentinels and dtype-specific behavior. Nullable extension types such as Int64, Float64 and boolean preserve a column's declared meaning while representing unknown entries with pd.NA. A comparison involving an unknown measurement can itself be unknown. That is useful information, but it needs a deliberate policy before becoming a selection mask.

In this lesson, the output records which scores are missing and which known scores are positive. Unknown scores are excluded from positive eligibility, but their original missingness remains visible in the table.

Construct the score column as Float64, then use isna to obtain an ordinary known missingness flag. Compare score with zero to produce a nullable boolean mask. Calling fillna(False) only on the eligibility mask says that unknown measurements do not qualify. It does not modify the original score column. Keep the missing flag and eligibility as separate boolean extension columns so their relationship can be inspected.

Trace the inputs missing, zero, negative two and positive three. Their missing flags differ from their positive flags. In particular, an excluded score is not necessarily a missing score, and the output preserves that distinction.

Filling the measurement itself with zero erases evidence about which rows were observed. The controlled mistake performs that replacement before producing flags. Its positive eligibility may look correct on a small sample, but its missing column falsely reports every row as observed. Another error is testing equality with pd.NA or relying on the truth value of a missing scalar. Use isna and notna for missingness instead.

Do not assume that every reduction treats missing data identically. A later groupby lesson distinguishes group size from the count of observed measurements and shows why a mean needs an explicit denominator.

Change eligibility to include observed zero scores as well as positive scores. Retain the original missing score values and the missingness flag. The small edit changes greater-than to greater-than-or-equal, while leaving the unknown-to-False mask policy intact.

Predict the three boundary rows: missing remains unknown in the score column and ineligible, zero becomes eligible, and a negative number remains ineligible. This isolates the new acceptance boundary without asking you to add a new pandas API. Keep the output columns and dtypes unchanged so the test failure points to the comparison rule rather than an unrelated representation difference.

Step 1: Include observed zero scores without replacing missing measurements. Step 2: Retain the working imports, input validation and table schemas. Change the stated policy, run the supplied practice cases, then explain the newly accepted or rejected boundary.

The solution changes gt(0) to ge(0). Acceptance inspects the nullable score values and both boolean columns, so merely returning a list of truth values cannot pass. The missing row must remain visibly missing even though the final eligibility mask contains no unknown entries. Empty input retains the same three-column schema.

The worked and solution suites include zero specifically to distinguish the two policies. They also include negative and missing values, preventing a broad fill or unconditional True result from passing. Explain why the missingness test and the threshold test answer different questions even though both eventually produce boolean output.

For an interview, explain how a nullable comparison differs from a two-valued Python boolean. Show the difference between a missing measurement, a zero measurement and a rejected measurement. Describe where fillna(False) belongs when unknown values should not qualify, and why applying fillna(0) to the source column can distort reporting.

Type the comparison and the explicit mask fill on the PC. The app asks about the resulting flags rather than evaluating code. Next, update an owning DataFrame safely under pandas 3 Copy-on-Write, where modifying a temporary selection does not update its parent.

Optional stretch — after the core task works: Add one independently reasoned boundary case and explain its expected value before running it.

Computer practice: https://applydjinn.com/typorithm/pandas-lab-20261106.zip. Install pinned dependencies in an isolated Python environment. Run verify.py, then change the working exercise baseline and run --practice pd-nullable. The real pandas tables, exact schemas, unchanged inputs, CSV parser options and owned in-memory SQL reconciliation use owned fixtures only. No production database, feed, account or paid service is used. This lab executes Python; the PC typing trainer and native reading progress do not execute it.