The first question we ask when someone describes what our API should do is: who is going to call it, and what are they going to do with the response? That question sounds obvious, but it shapes every consequential engineering decision in a provenance system. The answer for Musical AI is specific: performing rights organizations and music publishers, using the response to establish attribution claims and route royalty payments. That means the API has to fit into workflows that were built around existing rights administration formats, not designed by people who have ever thought about audio signal processing.
This post is a technical walkthrough of the engineering choices that follow from that constraint. It is not comprehensive documentation, but it is an honest account of why we made the choices we did.
What the Query Is Actually Asking
A provenance query for recorded music is asking a genuinely different question than an audio fingerprint lookup. Fingerprinting asks: is this audio file the same recording as a known reference track? It is an identity question. Provenance asks: which recordings contributed to the training of the model that generated this output? It is a lineage question.
The distinction has immediate technical consequences. A fingerprint lookup against a reference database can be performed directly against the submitted audio. A provenance query requires intermediate knowledge about the model's training history. You cannot answer a lineage question by only looking at the output. You need information about the process that produced it.
In practice this means a provenance API has two modes of operation. The first, and most tractable, is the case where the submitting party knows which AI model generated the track and can provide model metadata. The second is the forensic case: given only the generated audio, infer which training corpus is likely to have produced it based on spectral characteristics of the output.
Both modes are real. We built to support both, but the engineering complexity is significantly different between them, and the confidence scores in the response reflect that difference.
Response Schema Design for Rights Administration
The core decision in schema design was to make the response directly parseable by existing PRO intake systems without requiring a translation layer. That imposed a discipline on the response structure: use identifier types that rights administration systems already recognize, express ownership and weighting in formats that map to existing royalty calculation inputs, and avoid inventing novel data structures that create an integration burden on the receiving side.
The attribution result for a submitted track returns a list of source recording matches. Each match carries: the ISRC of the matched training recording, a confidence score expressed as a float between 0 and 1, a contribution weight representing the relative proportion of the match set attributable to this recording, and a rights holder identifier mapped from the ISRC via the catalog data we maintain.
The weighting structure deserves explanation. Attribution is not a binary match. A model trained on many recordings will produce outputs that reflect multiple inputs simultaneously. The contribution weights in a provenance response are designed to support proportional royalty routing: if three recordings each contributed roughly equally to the statistical patterns in a generated output, the weights reflect that and the royalty calculation can distribute accordingly. If one recording dominated the training influence on a specific output, the weights reflect that concentration. This maps more accurately to how audio influence actually works than a flat equal-split default would.
Handling Uncertainty Without Losing Usefulness
One of the hardest design problems in a provenance API is how to represent uncertainty without making the response useless. Rights administration workflows need deterministic answers. A royalty payment has to go somewhere. You cannot route a payment to "approximately 60% confident match with ISRC X."
Our approach is to decouple the analytical result from the administrative decision. The API returns confidence scores and contribution weights accurately, including low-confidence matches. The rights administration system consuming the response applies its own thresholds to decide which matches are actionable. We expose a recommended threshold field in the response to suggest a starting point, but the consuming system controls the cutoff.
This is the right separation of concerns. We know what the signal processing evidence supports. We do not know the PRO's internal policy for how strong the evidence needs to be before they initiate a royalty claim. Conflating those two decisions into a single yes/no response would either return unactionable noise or suppress valid claims by being too conservative. Keeping them separate gives each party the information they need to do their job.
The Catalog Reference Layer
The API is only as useful as the catalog it matches against. We maintain a reference catalog that includes ISRC records, ownership data, and spectral fingerprint indexes for recordings. Populating this catalog at sufficient scale and quality is an ongoing operational challenge, not a solved problem.
The architecture keeps the catalog queryable separately from the attribution pipeline. A caller can submit a fingerprint hash for a catalog lookup independently of submitting a full track for attribution analysis. This supports use cases where a rights holder wants to verify that a specific recording is represented in the reference catalog before submitting a track for analysis, which is a reasonable quality check before building an attribution workflow on top of the API.
Catalog coverage is the practical bottleneck for attribution quality. A provenance result can only identify training recordings that exist in the reference catalog. If a recording was included in a training dataset but is not indexed in our catalog, it will not appear in the attribution response. We represent this limitation directly in the API documentation and in the response metadata, which includes an estimated catalog coverage percentage for the genre and era of the submitted track.
Integration Into Existing Workflows
The decision to use JSON responses compatible with standard PRO intake formats was a deliberate integration-first choice. Rights administration software has long integration cycles and is not rebuilt for new data sources easily. If the attribution API required consuming systems to adopt a new data format, the adoption barrier would be prohibitive for most PROs.
We tested the response schema against the data structures used in common mechanical rights intake and performance rights reporting formats. The field names and identifier types in the response were chosen to minimize the translation work required on the receiving side. That is not accidental: it was a core design requirement from the beginning of the project.
The batch processing endpoint follows the same principle. Submitting a batch of tracks returns a job ID, and the status and results are retrievable asynchronously. This mirrors how bulk catalog ingestion works in existing PRO and publisher systems: submit a batch, poll for completion, retrieve results. We did not invent a new workflow; we matched the pattern that catalog teams already know.
What We Are Still Working On
Being direct about open problems is more useful than presenting a finished system. The forensic mode, inferring training provenance from audio alone without model metadata, produces lower confidence results than the model-aware mode. Improving that confidence without increasing false positives is active work. The catalog coverage problem has no clean technical solution at this stage; it requires ongoing catalog partnership and data acquisition. And the weighting model for contribution estimates is an approximation that will improve as we process more tracks and refine the underlying attribution methodology.
The API is designed to work inside existing rights management workflows. That is where it has to earn its usefulness, and that framing guides every design decision we make.
Stay Informed
