on disabling drivers that use non-Free firmware

[Richard M Stallman]

    Removing all the code in a driver, rather than simply disabling its
    requests for non-Free firmware, creates another major burden: any patch
    that touches files in that driver becomes an additional maintenance
    burden.

I don't see why.  Would you explain?

When a patch tries to change a file which is not present, that part
of the patch will be ignored, right?  So where's the burden?