10.1 Batch Processing Multiple IFC Files

Introduction

A batch script that already works is only half useful if it still needs a person to type out every file path by hand and watches it for the one file that breaks the whole run. Module 9 built a function that extracts wall quantities from a single file and a loop that calls it across a short, manually typed list of paths, and it named its own gaps honestly: no resilience when a file is bad, no record of what actually happened during a run, no way to point it at a folder and just let it work. This lesson closes those three gaps specifically, rather than reintroducing batch processing as if it were new.

Finding Files Automatically

A real batch job doesn't arrive with a pre-typed list of paths. It arrives as a folder full of files, and the first real improvement is having the script find its own inputs rather than being handed them one by one:

from pathlib import Path

def find_ifc_files(directory):
    return sorted(Path(directory).glob("*.ifc"))

Path(directory).glob("*.ifc") scans that folder for anything matching the pattern and returns them as Path objects, which work interchangeably with plain file path strings in the functions built across Module 9. Sorting the result matters more than it looks like it should: without it, the order files get processed in can vary between runs on some systems, which makes output harder to compare from one run to the next. This version only looks inside the folder given to it, not any subfolders; a real project's files might be nested deeper, and Path(directory).rglob("*.ifc") extends the same pattern recursively when that's needed, without changing anything else about how the result gets used.

Isolating Failure Per File

The batch loop built in 9.5 assumes every file in the list opens and processes cleanly. A real folder of files doesn't guarantee that, a corrupted export, a file that isn't actually valid IFC, a permissions problem, any of these stop the entire run the moment they're hit, taking down every result already collected for files that had nothing wrong with them. The fix is wrapping each file's processing in its own error handling, so one bad file gets recorded and skipped rather than ending the run:

def process_file(file_path):
    try:
        results = extract_wall_quantities(str(file_path))
        return {"file": str(file_path), "status": "success", "error": None}, results
    except Exception as e:
        return {"file": str(file_path), "status": "failed", "error": str(e)}, []

Catching Exception broadly here is a deliberate, specific choice for this stage, not a habit to carry into more targeted code: at the point a file is being opened, almost any failure means that one file couldn't be processed, and the correct response is the same regardless of which specific error caused it, log it, move on, keep the rest of the batch alive. Narrower exception handling has its place further into a script where different failures genuinely call for different responses; here, the goal is exactly one thing, isolating this file's failure from every other file's success.

Tracking What Happened, Not Just the Data

A batch run over many files needs a record of what happened to each one, independent of the extracted data itself. Two files returning zero results can mean two completely different things: one had no walls at all, a perfectly valid outcome, and the other failed to open entirely, a real problem worth knowing about. Conflating those into the same "empty result" tells nobody anything useful. Keeping a status record alongside the data, as process_file() above already returns, is what makes that distinction visible rather than lost.

A Complete Script

Putting this together: scan a folder, process every file found with isolated error handling, and write out both the extracted data and a run status report.

from pathlib import Path
import csv

def find_ifc_files(directory):
    return sorted(Path(directory).glob("*.ifc"))

def process_file(file_path):
    try:
        results = extract_wall_quantities(str(file_path))
        return {"file": str(file_path), "status": "success", "error": None}, results
    except Exception as e:
        return {"file": str(file_path), "status": "failed", "error": str(e)}, []

def run_batch(directory):
    file_paths = find_ifc_files(directory)
    all_results = []
    run_log = []

    for path in file_paths:
        status, results = process_file(path)
        run_log.append(status)
        all_results.extend(results)

    return run_log, all_results

def write_csv(rows, output_path):
    if not rows:
        return
    with open(output_path, "w", newline="") as f:
        writer = csv.DictWriter(f, fieldnames=rows[0].keys())
        writer.writeheader()
        writer.writerows(rows)

run_log, all_results = run_batch("/path/to/ifc_folder")

write_csv(run_log, "run_log.csv")
write_csv(all_results, "wall_quantities.csv")

succeeded = sum(1 for r in run_log if r["status"] == "success")
print(f"{succeeded} of {len(run_log)} files processed successfully")

Two files come out of this: one is the actual extracted data, unchanged in shape from 9.5. The other, run_log.csv, is new, a row per file naming whether it succeeded and, when it didn't, exactly what went wrong. Nothing here depends on extract_wall_quantities() being the specific function it is; the same process_file() wrapper and run_batch() structure works around any single-file extraction function built the way Module 9 built one.

What's Next

This script extracts data and records whether each file was readable. It doesn't check whether the data itself is any good, whether a wall's volume looks plausible, whether a required property is actually present, whether the file satisfies any real requirement at all. That's a genuinely different kind of check, sitting on top of extraction rather than being part of it, and it's exactly where the next lesson picks up: turning this batch runner into the shape of a basic automated QA pipeline.