Parse the boot index of each module the first time a path in it is read by Project516 · Pull Request #31 · Project516/java-web

Project516 · 2026-11-04T02:25:09Z

Every start parsed the whole file index in boot.pack: 20,901 entries, one per file of every module in the image. A start from a boot snapshot reads files from java.base only, so most of that work was wasted. It ran on every Node start of dist/node.js and on every page load in the browser worker.

Changes:

scripts/build-web.ts keeps directory lines and files outside the modules in /index.prefix. Each module's file lines go in their own block, /index/<module>.prefix.
PackIndex (src/browser/http.ts) is a Map that parses a module's block the first time get or has sees a path inside that module. listIndex parses it before listing a module's directories. Module directories stay in the main index, so the VM's check of which modules exist parses nothing.
HttpBackend.read and readRange look up the index before the lazy-pack and offset maps, since those entries appear when the module is parsed. Image files such as /java.base.jar are not indexed and still go straight to range reads.
addIndex scans each line with indexOf instead of split and a regex, and makes one image path string per module instead of one per file.

A full boot still reads every module-info.class, so it parses every module, as before.

Measured on the same machine:

Case	Before	After
Cold `parsePack`, plus the first `java.base` lookup (median of 10)	93 ms	29 ms
Index entries after a snapshot start	20,901	8,354
`dist/node.js` hello world from a snapshot (median of 10)	526 ms	500 ms

The "before" row for the hello world already uses the faster addIndex with the old single index, so against master the gain is somewhat larger.

Checks:

New test: module blocks are parsed on first lookup, listed, and give the same sizes, offsets and javac pack entries. Untouched modules stay unparsed.
pnpm typecheck passes, and pnpm test passes (139 tests).
The browser self-test passes every step, javac compile included, on a fresh and on a reused profile.

coderabbitai · 2026-11-04T02:25:24Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 25 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: dae50c53-f7e1-4bf0-8841-4a23e5c65092

Commits

Reviewing files that changed from the base of the PR and between 14700a8 and 7c91644.

Files selected for processing: 3

scripts/build-web.ts
src/browser/http.ts
test/image.test.ts

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

New Features
- Pack indexes now distinguish general files from module-specific files, with module entries available as paths within each module are accessed.
- File listings and ranged reads now use the relevant module’s index.
- Index compression preserves Unicode characters at line boundaries.

The build script now writes separate compressed indexes for module files. The browser pack reader stores module indexes and parses them on demand during path lookups and directory listings.

Changes: Module index flow

Layer / File(s)	Summary
Generate general and module indexes `scripts/build-web.ts`	Module file records are grouped into per-module indexes. The script writes those indexes and the general index as boot-pack entries using shared prefix compression.
Parse module indexes on demand `src/browser/http.ts`	`PackIndex` stores module index data and parses it on the first path lookup. `addIndex` scans index lines and records file metadata.
Use indexes for reads and listings `src/browser/http.ts`, `test/image.test.ts`	Read operations check indexed paths before lazy-pack access. Directory listings load the requested module index. The test checks that another module remains unparsed.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant listIndex
  participant PackIndex
  participant addIndex
  listIndex->>PackIndex: requests entries for a directory
  PackIndex->>addIndex: parses the requested module index
  addIndex-->>PackIndex: adds parsed paths and metadata

Possibly related PRs

Project516/java-web#4: Adds ImageBackend, which calls the pack-index functions and reads Pack.index.

Suggested reviewers: project516-review-bot

Merge Risk: Minimal · previous review 14700

Module indexes load on demand, with no actionable merge-blocking issue identified. Normal checks remain appropriate.

Pre-merge checks: 5

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: lazy parsing of each module's boot index on first path access.
Description check		The description directly explains the index format changes, lazy parsing behavior, performance impact, affected code paths, tests, and validation results.
Docstring Coverage		Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 3 functions across 3 files.

Finishing Touches

Generate docstrings

Commit to this branch
Create a new PR

Simplify code

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

_{A rabbit checks the module trails,
While compressed indexes ride the pails.
One path wakes its module’s page,
Another waits upon the stage.
The pack lists files, then hops away.}

Recent review info

Run configuration

Source mode: Auto

Repository: Unavailable

Branch: Unavailable

File: No configuration file from this repository applied

Commit: Unavailable

Layers: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: `aaae6f3d-fcbc-465f-ae6d-6079f96868a9`

Commits

Reviewing files that changed from the base of the PR and between `c34ccf7` and `14700a8`.

Files selected for processing: 3

`scripts/build-web.ts`

`src/browser/http.ts`

`test/image.test.ts`

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Recent review details

Context from checks skipped due to timeout: 6

GitHub Check: jtreg / run (4)

GitHub Check: jtreg / run (3)

GitHub Check: jtreg / run (1)

GitHub Check: jtreg / run (2)

GitHub Check: CodeRabbit

GitHub Check: test

Additional context used

OpenGrep (1.30.2)

scripts/build-web.ts

[ERROR] 152-152: A nonliteral argument is passed to exec/execSync. Identify the receiver: Node child_process methods execute shell commands, while regular-expression and database methods have different contracts. For shell execution, prefer execFile or spawn with an argument array.

(coderabbit.command-injection.exec-js)

Additional comments: 3

scripts/build-web.ts: 1

`132-133`: LGTM!

Also applies to: 152-158, 192-203, 205-206

src/browser/http.ts: 1

`10-10`: LGTM!

Also applies to: 40-40, 44-48, 53-87, 89-90, 92-93, 95-99, 102-102, 105-124, 212-213, 236-237, 296-296, 299-299

test/image.test.ts: 1

`77-94`: LGTM!

project516-review-bot

The PR introduces lazy parsing of module index blocks in boot.pack. Instead of parsing all 20,901 entries on every start, each module's index (/index/.prefix) is parsed only when a path in that module is first accessed via get/has or listed via listIndex. The build script now writes separate prefix-coded index files per module. PackIndex extends Map and overrides get/has to trigger loading. HttpBackend checks the index before lazy/offset maps. A new test verifies lazy parsing works correctly.

_{review-bot, model nvidia/nemotron-3-ultra-550b-a55b:free, verdict approve}

project516-review-bot

The PR introduces lazy parsing of module index blocks in boot.pack. Instead of parsing all 20,901 entries at startup, each module's index (/index/.prefix) is parsed on first access via PackIndex.load(). The build script now writes per-module index blocks, and HttpBackend.read/readRange/listIndex trigger loading before lookups. Tests pass and benchmarks show significant startup improvement.

_{review-bot, model nvidia/nemotron-3-ultra-550b-a55b:free, verdict approve}

project516-review-bot · 2026-11-04T03:01:32Z

+    const raw = this.modules.get(module);
+    if (!raw) return;
+    this.modules.delete(module);
+    addIndex(this.pack, new TextDecoder().decode(raw), true);


Optional: creating a new TextDecoder on every module load allocates unnecessarily. Consider reusing a single decoder instance (e.g., a static const or a field on PackIndex).

Done in a follow-up: the pack reader now shares one module-level TextDecoder (commit 5f9bb23, in the next pull request). There are at most about 30 module loads per VM, so this is tidiness more than speed.

Parse the boot index of each module the first time a path in it is read

14700a8

project516-review-bot Bot approved these changes Nov 4, 2026

View reviewed changes

coderabbitai Bot approved these changes Nov 4, 2026

View reviewed changes

Project516 enabled auto-merge (squash) November 4, 2026 02:36

Merge branch 'master' into p516/lazy-module-index

7c91644

project516-review-bot Bot approved these changes Nov 4, 2026

View reviewed changes

Project516 merged commit be8af85 into master Nov 4, 2026
12 checks passed

Project516 deleted the p516/lazy-module-index branch November 4, 2026 03:20

Project516 mentioned this pull request Nov 4, 2026

List pack index directories from a map built once, which makes javac's first compile about a quarter faster #34

Merged

coderabbitai Bot mentioned this pull request Nov 4, 2026

Run on a jlinked image-mode JDK instead of the exploded layout #37

Merged

Navigation Menu

Repository navigation

Parse the boot index of each module the first time a path in it is read - #31

Parse the boot index of each module the first time a path in it is read#31
Project516 merged 2 commits into
masterfrom
p516/lazy-module-index

Project516 commented Nov 4, 2026

Uh oh!

coderabbitai Bot commented Nov 4, 2026 •

edited

Loading

Review limit reached

Summary by CodeRabbit

Sequence Diagram(s)

Uh oh!

project516-review-bot Bot left a comment

Uh oh!

project516-review-bot Bot left a comment

Uh oh!

project516-review-bot Bot Nov 4, 2026

Uh oh!

Project516 Nov 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Project516 commented Nov 4, 2026

Uh oh!

coderabbitai Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Summary by CodeRabbit

Sequence Diagram(s)

Uh oh!

project516-review-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

project516-review-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

project516-review-bot Bot Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

Project516 Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Nov 4, 2026 •

edited

Loading