Where the results came from

Training and evaluation records

Paths below start at the workspace root. The original records stay in the research workspace while the local web preview serves this index.

Scratch checkpoint candidates

Paths are relative to outputs/tagalog-scaling-v0.19. Each entry includes its SHA-256 hash.

The 256-token candidates use the original 2K tokenizer. Their frozen inputs are listed in paired-run-v11/inputs/manifest.json. All five input hashes matched the saved files. The 512-token copy candidates use the 4K tokenizer and share the starting checkpoint and input hashes documented in ab-512/supervised-copy-v1/plan.json. Both copy variants passed zero of eight full chats in that study.

Frozen corpus for the 256-token runs

Counts come from the explicit split field in the two frozen JSONL files. The snapshot also contains a separate evaluation packet and a Filipino diagnostic. Find their hashes in paired-run-v11/inputs/manifest.json. These counts describe the archived training inputs. Source attribution and redistribution status still need review before release.

Source labels in the frozen 256-token snapshot

The 84 prose passages comprise 34 explicitly labeled original or AI-authored passages and 50 others grouped as storybook (35), journalism (14), or Wikipedia (1). These categories describe the archived metadata. For all 15 dialogue rows with empty source fields in the frozen snapshot, the record IDs and complete message arrays matched reviewed intake files. The 11 synthetic dialogues match synthetic-topic-rewrites-01.jsonl through -03.jsonl. The four VOA translations match translated-voa-002.jsonl, -003.jsonl, and -pilot-01.jsonl. The translations and externally sourced prose still require source-specific redistribution review.

The redacted dataset candidate excludes four Voice of America Learning English source families because redistribution permission for the adaptations could not be independently verified. The earlier public staging export already omitted them. The historical 256-token training snapshot contains all four adaptations. Existing weights and reported results belong to their original runs, and the redacted candidate is separate from those historical snapshots.

Recorded license and attribution details

The frozen prose metadata labels 14 journalism excerpts CC BY 3.0 and one Wikipedia excerpt CC BY-SA 4.0. The 35 storybook entries carry CC BY 3.0 in their separate license URL field, with credits and source links for all 13 titles. All 13 linked publisher pages display CC BY 3.0 and match the archived translator credits.

Voice of America Learning English published all four transcripts in the Conversation sections of its Let’s Learn English – Level 1 lessons. The dates below are those displayed on the lesson pages. VOA credits certified American English teachers with designing the course, but the reviewed lesson pages did not identify individual transcript writers, credit a third party for the dialogue text, or provide a transcript-specific copyright notice. The course listing supports VOA publication. It leaves exclusive production and U.S. government-work status unverified for each script. VOA course listing

VOA’s terms identify exclusively VOA-produced material as public domain and ask users to give credit. They reserve rights for third-party material. If a transcript is confirmed to fall under the public-domain policy, U.S. copyright would not restrict translation, adaptation, copying for training, or redistribution of that transcript. The reviewed sources do not establish that condition or provide separate transcript-specific permission for these uses. The intake records’ approval flags document project decisions and do not establish rights clearance. VOA terms of use · U.S. Copyright Office, 17 U.S.C. §§ 101, 105–107

Private research and training require their own assessment. Research can qualify as fair use under U.S. law depending on the circumstances. Reporting aggregate findings and linking to the lessons can proceed independently of distributing adapted dialogue. Hold these four rows out of any public dataset export until exclusive production is confirmed or applicable permission is documented. The audit does not establish worldwide reuse rights.

Storybook attribution

The list credits the original text and the Tagalog translators. The frozen records also retain the original story URL and whether each excerpt was copyedited. All records link to the CC BY 3.0 license.

• Si Anansi at ang Dunong · 2 excerpt(s) · Ghanaian folktale · Translation Arlene Avila
• Bagong buhay · 2 excerpt(s) · Ursula Nafula · Translation Arlene Avila
• Si Nozibele at ang Tatlong Hibla ng Buhok · 4 excerpt(s) · Tessa Welch · Translation Arlene Avila
• Ang Paghihiganti ng Pastol ng Pulut-Pukyutan · 9 excerpt(s) · Zulu folktale · Translation Karla Comanda
• Magaling si Zama! · 2 excerpt(s) · Michael Oguttu · Translation Karla Comanda
• Ang buwayang gutom · 1 excerpt(s) · Christian G. · Translation Karla Comanda
• Si Tingi at ang mga Baka · 1 excerpt(s) · Ingrid Schechter · Translation Karla Comanda
• Pagbibilang ng mga Hayop · 1 excerpt(s) · Zanele Buthelezi, Thembani Dladla, Clare Verbeek · Translation La Trinidad Mina
• Parusa · 1 excerpt(s) · Adelheid Marie Bwire · Translation Arlene Avila
• Sabi ng ate ni Vusi · 5 excerpt(s) · Nina Orange · Translation Arlene Avila
• Paluwas sa malaking lungsod · 4 excerpt(s) · Lesley Koyi, Ursula Nafula · Translation Arlene Avila
• Mahilig akong Magbasa · 1 excerpt(s) · Letta Machoga · Translation La Trinidad Mina
• Mga Batang Sera · 2 excerpt(s) · Southern African Folktale · Translation Arlene Avila

SEA-LION and Gemma development counts

Development evaluation: 20 six-exchange conversations per configuration, with 120 replies and 72 rated state-tracking checks per configuration. Reviewers scored the runs separately without blinding. The results describe a small sample.

What the archive still needs

Each public checkpoint needs its own manifest with the exact tokenizer, input data and parent weights. The public dataset package needs a source-level attribution and redistribution review. The 20-chat Gemma result is a development result with three full six-exchange passes.