Research
Summary
We're testing whether some people can reliably predict a song's potential before anyone else. Only blind bets made in Play A&R count, and a good run has to repeat on new songs or it's treated as luck. These rules were locked and registered before any results existed. If the idea fails, we'll publish that too.
TINPAN RESEARCH PROTOCOL v1 (sections 2 to 7 of "Tinpan pre-registration and endpoint policy v2", 21 October 2026)
Hypothesis, arms, unit of analysis
H1 (skill exists): listeners ranked in the top quarter by matched-baseline delta on period A have a positive delta on period B, with a 95% interval excluding zero. Period A and period B are disjoint sets of songs and disjoint calendar windows. Skill that does not persist is noise.
H2 (surprise predicts outcomes): in the mixed-effects model stated in section 8.1 of the white paper, the surprise-weighted bet term is positive with an interval excluding zero after conditioning on raw bet volume and serve count.
Arms. Every call carries an arm: blind (made in Play, no title, writer, rank or bet count visible) or informed (made on a room, chart or song page). Only blind calls enter H1 and H2. Informed calls are reported as a separate association, not a causal estimate, until the planned within-song randomization runs.
Unit of analysis is the call: one listener, one song, one timestamp, one arm, one social state frozen at the call. A listener's first call on a song is canonical. A repeat call on a song the listener had already heard anywhere on Tinpan is kept in the record but excluded from H1 and H2; the count of such calls is reported.
Songs. Brief entries are the primary song set, because they are written for the brief and have no prior public exposure. Room drops are the secondary set, reported separately, with the fingerprint-at-upload check as the prior-release filter once it ships.
Endpoints
Primary endpoint is a desk action recorded as desk-blind: an inquiry, a hold or a deal on the song by a verified desk whose view, at the moment it acted, showed no rank, bet count or tier. The white paper already logs what the desk could see; this page makes desk-blind the condition for the primary endpoint. Desk-blind means the brief board showed no crowd signal when the desk acted (inquiry, hold or deal from that board with the crowd switch off); it does not claim the desk never saw a count elsewhere on the site, and the record says which. A desk action taken with crowd signal visible is a secondary endpoint, reported separately.
Secondary endpoint is chart rise under endpoint policy v2 (section 7): the song gains at least 15 percentile ranks within its station, or enters the top quarter, and holds that position across two consecutive Monday freezes within the horizon. The current game rule (any rise, off-chart counted as 101) stays as the game score and is not evidence for H1.
Tertiary endpoint is any external outcome reported from outside Tinpan (cut, release, sync, consumption on another service), tagged by whether a Tinpan brief or referral was in the path. Only outcomes with no Tinpan path count toward the claim that judgment predicts the market rather than creates opportunity.
Horizon T for chart rise is set from the empirical time-to-rise distribution once 100 rises exist, per the paper's censoring rule. Until then, calls without an elapsed horizon are censored, never forced to a miss.
Baseline: the matched opportunity set
A listener is scored against the songs they were actually served, not against a global base rate. For listener h, let S_h be the set of songs served to h in the blind arm with an elapsed horizon, and B_h the subset h bet on. The skill delta is the hit rate on bets minus the hit rate on everything served:
\Delta_h = \frac{1}{|B_h|}\sum_{m \in B_h} \mathbb{1}[y_m = 1] \;-\; \frac{1}{|S_h|}\sum_{m \in S_h} \mathbb{1}[y_m = 1]
This controls for which songs, rooms and chart states each listener saw, and for how often they bet versus passed. A listener who bets on everything has a delta of zero by construction. A listener who bets on five songs and hits four is scored on what they passed, not only on what they picked.
Significance of a single delta is tested against a permutation null: draw |B_h| songs at random from S_h, many times, and ask how often the random delta meets or beats the observed one. A listener's delta counts as positive at p < 0.01 under that null. Two companion numbers are reported with every delta: coverage (|B_h| / |S_h|) and the share of served songs that rose (|S_h| hit rate), so a high delta on a tiny or easy opportunity set is visible as such.
For the machine hierarchy, the same delta is computed for each baseline player (random at the station base rate, volume-following, audio embedding with a linear probe, aggregate crowd) on the same served sets, so humans and machines are compared on identical opportunity.
Sample size, look rule, failure criteria
The white paper's power formula gives roughly 100 resolved blind bets per listener to detect a 10-point edge over a 10% base rate. That is the floor for naming any individual listener skilled, and for the Superlistener rung. Nobody clears it in 2026.
So the first analysis is declared a pilot in advance. The pilot opens when two briefs have resolved and at least 40 listeners each have 30 or more resolved blind bets. The pilot reports population-level results only: the distribution of deltas, H2, the blind-versus-informed association, and Krippendorff's alpha on shared songs. It does not name skilled individuals.
H1 is tested when at least 20 listeners each have 100 or more resolved blind bets spread across two disjoint periods. Nobody looks at per-listener period-B outcomes before that gate; the chart and the game run on their own rule meanwhile.
Failure is stated now:
- H1 fails if the top-quarter cohort's period-B delta interval includes zero, or the Spearman correlation of deltas across periods is below 0.3 or has an interval including zero.
- H2 fails if the surprise term's interval includes zero after conditioning on volume.
- The precondition fails if, by the H1 gate date, fewer than 20 listeners have reached 100 resolved blind bets. That is reported as the market not sustaining the instrument, not as evidence about judgment.
A failed test is published as a failed test. The grading rule and the baseline are not changed after the look; a better rule is a new version applied prospectively.
Freezing the protocol
After the build check, the text of sections 2 to 7 is exported as plain text, hashed with SHA-256, and stored in a table research_protocols with columns: version, hash, frozen_at (server timestamp), text. The row is insert-only, same pattern as agreement text: a new version is a new row, never an edit. The hash and date are shown on a public page (About or a Research line in the footer), and the same text is registered on the Open Science Framework (osf.io), which timestamps it independently of Tinpan.
The freeze happens before the first brief closes, whichever brief that is. The row must exist before any outcome does.
Any published result cites the protocol version and hash it was run under. If the analysis departs from the frozen text, the departure is stated in the result, and the frozen text is not changed.
Endpoint policy v2
The game keeps grading on v1 (any rise, off-chart as 101). v2 is a second grading rule, stored as a new designation row and applied prospectively from its effective date, so both resolutions exist side by side on every call and neither rewrites the other. This is the versioned-policy pattern already in the architecture.
| Field | v1 (game) | v2 (science) |
| Rise | Any later issue places the song strictly above the position frozen on the call | Gains 15 or more percentile ranks within its station, or enters the top quarter of the station |
| Off-chart | Position 101; entering at 100 counts | Entering counts only if the entry position is within the top quarter |
| Persistence | None; one issue is enough | The qualifying position holds across two consecutive Monday freezes |
| Horizon | Set from observed time-to-rise | Same horizon; calls still inside it are censored |
| Void | Song withdrawn | Song withdrawn, or station had fewer than 20 live songs at the call |
| Feeds | Points, record, ladder | H1, H2, machine hierarchy; never points |
Percentile rank is the song's position divided by the number of live songs in the station on that issue, so a station of 30 songs and a station of 300 are graded on the same scale. The 20-song floor exists because a quarter of a 12-song station is three songs.
Build check 21 Oct: issues store positions and a global population, not station size. v2 computes station size per issue from the frozen positions through the room-to-station map, and stores it going forward.