Companion piece to last week’s Berkshire-AI-exclusion read — one rung deeper on the same evidentiary thread.
Executive summary
AI-induced loss is not yet well-enough understood for carriers, regulators, or auditors to assess on traditional terms. The reason is partly that the data the model touches is data the evaluator cannot see — either because the evaluator does not have direct access (privacy, regulation, contract) or because the data’s holder cannot be allowed to share it with the evaluator (HIPAA, GDPR, trade-secret, national security). The methods to evaluate AI systems under these constraints already exist. They are mature in adjacent engineering disciplines: privacy-preserving cryptography, hardware-attested confidential computing, statistical sampling, formal verification, and federated analytics. What is missing is the composition — the integrated stack that satisfies a carrier’s underwriting attestation, a regulator’s compliance audit, an IRB’s data-use check, and a patent-prosecution defensibility record from the same operational evidence. Building that composition is the engineering homework of the next eighteen months.
For the busy reader: the methods to evaluate AI without seeing the underlying data exist today and are partly in production — federated evaluation, secure aggregation, differential privacy, confidential computing inside hardware-attested enclaves, and zero-knowledge proofs of model evaluation. What is missing is the composition. The engineering homework of the next two years is figuring out which subset of these methods, in which sequence, with which cryptographic and procedural binding, satisfies an underwriter and an FDA inspector and an IRB and a patent prosecutor at the same time.
The two flavors of can’t-see-the-data
Two different scenarios both produce evaluation-without-data, and they require different tools.
Data the evaluator cannot see. A patent examiner reviewing an AI-assisted invention defense cannot subpoena the proprietary training corpus (fn. 1). An underwriter cannot demand the policyholder’s production data to verify control firing. A regulator certifying a financial-services model cannot exfiltrate the bank’s consumer-credit data. The data exists; the evaluator simply does not have a path to it.
Data the data-holder cannot be allowed to share. A clinical-AI consortium where each hospital’s data is HIPAA-protected and cannot leave the institution’s firewall. A multi-national AI deployment where each region’s data is GDPR-protected. A national-security AI system where the operational data is classified. The data-holder would share if they could; they are constrained by law, contract, or policy from doing so.
Both produce the same evaluation challenge. The privacy-preserving cryptography literature treats both. But the operational, regulatory, and contractual scaffolding differs — and the choice of method depends on which side of the constraint you are on. Concrete examples:
Healthcare consortium: Boston Children’s, MGH, and Dana-Farber want to train and evaluate a federated diagnostic model. Each hospital’s patient data is HIPAA-protected; no hospital can share data with the others. Solution class: federated learning + secure aggregation. Patent-prosecution defense: An AI-assisted invention is challenged on §101 grounds. The applicant needs to demonstrate the inventive-step contribution of a human reviewer without exposing trade-secret training data to the patent examiner. Solution class: sealed-record evidence + attestable model-version pinning. Cross-vendor industrial federation: Five competing industrial-AI vendors want to jointly benchmark performance on real customer data without exposing competitive information. Solution class: confidential computing inside hardware-attested enclaves + zero-knowledge proofs.
Three different problems. Three different method compositions. All inside the same broad category of evaluation-without-data.
The method stack that exists right now
Three method families converge on this problem. Each has a real production deployment base and an accumulating practitioner literature. The composition is what is missing — not the parts.
Privacy-preserving evaluation
Federated evaluation. A central node ships an evaluation protocol to each participating site. Each site runs the evaluation locally on its private data and returns only aggregate metrics. Raw data never leaves the site. Google’s federated-learning stack has been running this in production since 2017; Apple’s on-device intelligence relies on the same pattern at consumer-device scale.
Secure aggregation via multi-party computation. Cryptographic protocols where each site encrypts its evaluation outputs so that only the aggregate (sum, mean, weighted average) is decryptable; individual site contributions stay encrypted (fn. 2). Production-deployed in federated keyboard learning, federated healthcare analytics, and consortium ML training.
Differential privacy (DP). Differential privacy is a formal mathematical guarantee that adding or removing any single individual from a dataset will not materially change the output of an analysis (fn. 3). Calibrated noise is added to released metrics so that aggregate measurements carry formal ε-DP bounds. Adopted at U.S. Census scale (2020 Census released under DP); at Google scale (RAPPOR, Privacy Sandbox); and at Apple scale (on-device DP for usage telemetry). For AI evaluation: a model’s released accuracy, F1, or AUC can carry DP bounds without exposing individual training examples.
Homomorphic encryption. Evaluations run on encrypted data; results decrypted only by the data owner. Fully homomorphic encryption (FHE) supports arbitrary arithmetic on ciphertext but is computationally heavy. Partial / leveled homomorphic encryption is more practical and is used in production at Microsoft (SEAL) and IBM (HElib) for specific evaluation patterns.
Confidential computing inside Trusted Execution Environments (TEEs). Hardware-attested enclaves — Intel SGX, AMD SEV, AWS Nitro Enclaves, Azure Confidential VMs — where data and computation run inside a hardware-rooted boundary the cloud operator cannot see. The commercial use case is broader than the academic literature usually surfaces: financial-services firms run confidential analytics on competitor trading data; advertising networks run audience-overlap measurements without exposing customer lists; supply-chain consortia run inventory benchmarks without disclosing individual-firm volume. The Confidential Computing Consortium is the standards-track home.
Zero-Knowledge Machine Learning (ZK-ML). ZK-ML uses zero-knowledge proofs to demonstrate that a model evaluation passed specified checks (e.g., “the deployed model achieves ≥0.85 AUC on the sealed test set”) without revealing the data, the model details, or the intermediate computation. ZK-SNARK and ZK-STARK constructions have moved from research to pre-production for serious applications. EZKL and Ingonyama are the active engineering houses (Modulus Labs was a third entrant until its 2024 acquisition by Tools for Humanity).
Synthetic data with formal fidelity bounds. Sites generate privacy-preserving synthetic surrogates of their data using differentially-private generators — DP-GANs, DP-VAEs, DP-diffusion. Evaluation runs on synthetic data with provable bounds on how well the synthetic distribution approximates the real one (fn. 4).
Engineering V&V tradition translated
A reasonable critic asks: is the V&V tradition itself a big soup? The seven regimes named in the Berkshire piece (FDA SaMD, SR 11-7 (now SR 26-02), NASA-STD-7009B, MIL-STD-3022, IEEE 1012, DO-178C, NRC 10 CFR Part 50 App B) cover different industries, different rigor scales, and different audit cadences. The answer is that they share a common operational pattern even though their detailed clauses differ — and the pattern translates to AI evaluation almost without modification.
Sealed test sets. The auditor or qualifying body holds a test corpus. The deployer evaluates against the sealed set and returns results without ever seeing the inputs. This is the operational backbone of DO-178C aviation certification — every line of safety-critical avionics software is qualified against test cases the developer does not specify (fn. 5).
Acceptance sampling protocols. Statistical sampling plans with confidence bounds, codified in ISO 2859 and ANSI/ASQ Z1.4 for manufacturing quality control. The auditor specifies a sampling plan, the deployer executes it on its private data, and aggregate statistics with confidence intervals are returned. The decision rule (accept / reject) is independent of who saw the data.
Property-based testing and formal verification. Instead of evaluating against data, prove the model satisfies specified properties (monotonicity, fairness constraints, bounded output ranges, safety envelopes). Marabou (Stanford) and α,β-CROWN (Princeton/IBM) are production-grade neural-network verifiers. For high-stakes deployments, formal verification is sometimes the only credible evaluation method — the data is irrelevant if the property holds across the input domain.
Statistical disclosure limitation. Classical methods from census and survey research: k-anonymity, l-diversity, t-closeness. The HIPAA Safe Harbor de-identification standard codifies a version of this. Useful as a baseline; insufficient by itself for modern AI evaluation (fn. 6).
Model cards and datasheets. Formal documentation that captures evaluation summaries plus dataset characteristics without releasing raw data. The practitioner conclusion: a deployed AI system without a current model card and a current datasheet for its training data is structurally unable to provide the multi-audience evaluation evidence underwriters and regulators will eventually require. Hugging Face and Google have institutionalized model cards as a publication-grade artifact (fn. 7).
Composition stacks across industries
Healthcare. The OHDSI / OMOP Common Data Model lets multiple hospital data warehouses participate in standardized analyses without raw-data sharing. MELLODDY is the cross-pharma federated-learning consortium with cryptographic guarantees. NIH SRA/DUA (Subject Research Agreement / Data Use Agreement) architectures provide the contractual scaffolding. Academic medical centers in the OHDSI collaborator network and the N3C National COVID Cohort Collaborative commonly run multi-institutional analytics through these stacks — the practical state of cross-institutional firewall “openness” varies, but the framework is real.
Financial services. Cross-bank federated analytics for systemic risk modeling, fraud detection, and stress testing have been running under similar architectures for several years — see BIS Financial Stability Institute Insight 60 for an industry-level survey, and the Federal Reserve Financial Stability Report series for the regulator-side view. The Federal Reserve’s Cross-Country and Multilateral Studies, the FDIC’s deposit-insurance modeling, and inter-bank ML consortia all use sealed-test-set + federated-evaluation patterns.
Industrial IoT and energy. The Industry IoT Consortium has piloted federated-learning approaches for predictive maintenance across competing vendors; the academic literature on industrial federated learning is collected at the arXiv cs.LG section with substantial published work on cross-vendor preventive-maintenance federation. PJM Interconnection coordinates demand-response analytics across more than 1,000 member utility and retail-load entities and publishes aggregate market clearing data without exposing customer-level consumption; ISO New England’s Demand Resources program follows a similar pattern. The energy-sector composition stack adds EPRI’s Grid Modernization Playbook work on cross-utility federated learning for predictive maintenance and forecasting.
Government and inter-agency. The U.S. Census Bureau’s adoption of differential privacy for the 2020 Census — the Disclosure Avoidance System — remains the largest production deployment of formal ε-DP at scale anywhere in government. The methodology is documented down to the ε budget per query class. Beyond Census, the federal evaluation pattern shows up across NIST AI RMF profile pilots, the DOJ Civil Rights Division Special Litigation Section pattern-of-practice analytics across multi-jurisdiction policing data (NAACP LDF backgrounder; DOJ source removed), and the Treasury Department’s cross-agency anti-money-laundering analytics under FinCEN Section 314(a) information-sharing protocols that combine FinCEN, IRS, and bank-supervisor records without raw-data sharing. These are early-production deployments. They share the same composition pattern as the healthcare and industrial stacks above.
The composition vocabulary differs across industries. The underlying methods do not.
The composition problem
None of the methods alone satisfies the multi-audience attestation requirement that high-stakes AI deployment actually faces.
A carrier-grade attestation needs the audit trail an underwriter can verify. A 21 CFR Part 11 compliance check needs the contemporaneous record of actual controls firing that an FDA inspector can request (fn. 8). An IRB data-use compliance check needs the institutional-consent chain back to each patient’s authorization. An inventor-side defensibility record for AI-assisted invention prosecution — particularly under §101 challenges, where the question is whether the human inventor’s contribution can be demonstrated separately from the AI’s synthesis — needs the natural-person attribution chain at the moment of recognition (fn. 9). Each method family above does part of this. None does all of it.
The composition is what is missing.
The engineering work that actually matters in the next two years is figuring out which subset of methods, in which sequence, with which cryptographic and procedural binding, satisfies an underwriter and an inspector and an IRB and a patent prosecutor at the same time. That is a real engineering project. It is not waiting for the next OECD principle or the next EU directive or the next NIST framework iteration. It is waiting for a standards body to host the working group, a few institutional sponsors to fund the demonstration, and a few engineering teams to publish reference compositions other companies can adopt.
What the practitioner discipline looks like
For executives building AI deployment in regulated industries today, the practitioner-side discipline has three moves.
Choose a method composition before the carrier asks. Pick a default stack — federated evaluation + secure aggregation + TEEs + sealed test sets + model cards is one reasonable composition for healthcare-AI; substitute homomorphic encryption + statistical disclosure limitation + property-based testing for higher-rigor financial-services contexts — and run it as production discipline. The carrier-attestation answer will be on the shelf when the underwriter asks for it. The IRB-compliance answer will be there when the audit happens. The prosecution-defense answer will be there when the §101 challenge arrives.
Build the cryptographic-and-procedural binding once, not per audience. The same evaluation evidence should be capable of producing the underwriter attestation, the regulator submission, the IRB exhibit, and the patent-prosecution record. The composition cost is paid once; the audience-specific reports are templated views over the same underlying evidence. This is exactly how the cyber-insurance attestation infrastructure (ISO/IEC 27001 reports, SOC 2 Type II audits) should work today — one underlying audit, many audience-specific deliverables.
Treat method composition as authored IP. A company that publishes its evaluation-composition pattern — how it combines federated evaluation, DP bounds, TEEs, sealed test sets, and model cards into a unified attestation pipeline — is contributing to the standards-body conversation while also documenting its own defensibility. The next ISO/IEC 42001 revision, the next NIST AI RMF profile, the next IEEE P7000-series standard will be drafted from exactly these kinds of published compositions. Engineering organizations that move first set the templates the rest of the market will use.
The methods are there. The composition is the engineering homework of the next eighteen months.
Footnotes
(fn. 1) “§101 challenges” refers to challenges under 35 U.S.C. § 101, the U.S. Patent Act provision defining patentable subject matter. After Alice Corp. v. CLS Bank Int’l, 573 U.S. 208 (2014), https://supreme.justia.com/cases/federal/us/573/208/, §101 became the most common ground for invalidating software-and-data-driven patents. AI-assisted inventions face heightened §101 scrutiny because the question of whether the “inventive step” reflects human contribution (patentable) or AI synthesis (not yet recognized as patentable) is doctrinally unsettled. The defensibility record at issue is the contemporaneous evidence that the human inventor’s contribution at the moment of recognition is distinguishable from the AI’s output. For a lay audience: the patent system has always required a human inventor; the new challenge is proving that the AI tool was an aid to the human, not the inventor itself.
(fn. 2) Multi-party computation (MPC) is a class of cryptographic protocols where multiple parties compute a joint function on private inputs without any party learning the others’ inputs. The aggregate result is decryptable; individual contributions are not. Survey reference: Yehuda Lindell, Secure Multiparty Computation, 64 Comm. ACM 86, 86–96 (2021), https://eprint.iacr.org/2020/300. Production-deployment reference (federated keyboard learning, federated healthcare analytics, consortium ML training): Keith Bonawitz et al., Practical Secure Aggregation for Privacy-Preserving Machine Learning, in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ‘17) 1175 (2017), https://research.google/pubs/practical-secure-aggregation-for-privacy-preserving-machine-learning/.
(fn. 3) Foundational work by Cynthia Dwork (Microsoft Research) and Aaron Roth. The “ε” parameter quantifies the privacy budget — smaller ε means stronger guarantee and more noise; larger ε means looser guarantee and less noise. Full mathematical treatment: Cynthia Dwork & Aaron Roth, The Algorithmic Foundations of Differential Privacy, 9 Foundations & Trends in Theoretical Computer Sci. 211 (2014), https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf. Practitioner-grade explainer: NIST, Privacy Engineering Collaboration Space, https://www.nist.gov/itl/applied-cybersecurity/privacy-engineering.
(fn. 4) Synthetic data with formal fidelity bounds works when (a) the synthetic distribution closely approximates the real distribution for the metrics being evaluated, and (b) the bounds on approximation error are explicit and quantified. Both conditions are non-trivial. The MIT Synthetic Data Vault project is the leading open-source practitioner toolkit. The fidelity-bound limitation is why synthetic data is a complement to, not a replacement for, the cryptographic methods above.
(fn. 5) DO-178C / ED-12C structures software-certification rigor across five Design Assurance Levels (DAL-A through DAL-E). DAL-A applies to catastrophic-failure-consequence systems; DAL-E applies to no-effect systems. The structural insight is that V&V rigor scales to consequence; AI evaluation regimes need an equivalent integrity-level taxonomy that the current discourse has not yet produced.
(fn. 6) Statistical disclosure limitation (k-anonymity, l-diversity, t-closeness) is necessary but insufficient because: (a) modern AI systems can re-identify individuals even from heavily-redacted datasets through model-inversion attacks; (b) k-anonymity does not protect against differential attribute disclosure; (c) the static-table assumptions of classical SDL do not map cleanly to model-output disclosure risk. SDL is therefore a baseline floor for what should not be released, not a ceiling for what evaluation-without-data can demonstrate.
(fn. 7) Margaret Mitchell et al., Model Cards for Model Reporting, in Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ‘19) 220 (2019), https://arxiv.org/abs/1810.03993; Timnit Gebru et al., Datasheets for Datasets, 64 Comm. ACM 86, 86–92 (2021), https://arxiv.org/abs/1803.09010.
(fn. 8) 21 CFR Part 11 is the FDA regulation governing electronic records and electronic signatures for FDA-regulated products. The “request” standard is more accurate than “subpoena” for the inspector relationship — the inspector typically requests records under the inspector’s authority; subpoenas come into play when records are contested or refused.
(fn. 9) See fn. 1 for §101 background. The natural-person attribution chain at the moment of recognition is the contemporaneous documentation that a specific human reviewed a specific AI output and made the inventive judgment to use it.