diff --git a/AGENTS.md b/AGENTS.md index 923c5cd68..34a0ef731 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -187,3 +187,139 @@ run: * New behavior must be documented through tests. + +--- + +## 7. LSP Structure and Build Pipelines + +This repository has multiple entry points that may trigger compilation/build behavior: + +* **Language Server runtime** + `de.peeeq.wurstio.languageserver.*` +* **LSP build request** + `de.peeeq.wurstio.languageserver.requests.BuildMap` +* **CLI compiler entry point** + `de.peeeq.wurstio.Main` +* **CLI map build request** + `de.peeeq.wurstio.languageserver.requests.CliBuildMap` + +### LSP architecture (high-level) + +* `WurstLanguageServer` wires LSP protocol handlers. +* `LanguageWorker` serializes requests and file-change reconciliation. +* `ModelManagerImpl` owns project model state (wurst files, dependencies, diagnostics). +* User actions like build/start/tests are implemented in `languageserver.requests.*`. + +### Build-map pipeline (centralized) + +Map build behavior is centralized in: + +* `MapRequest.executeBuildMapPipeline(...)` + +Both: + +* `BuildMap` (VSCode/LSP build command), and +* `CliBuildMap` (CLI `-build`, used by grill) + +must use that shared backend flow. + +This pipeline handles: + +1. map/cached-map preparation +2. script extraction/config application +3. compilation (Jass/Lua) +4. script + map data injection (including imports/w3i) +5. final output map write + MPQ compression finalization + +### Lock handling policy + +* `BuildMap` (LSP/UI) may use interactive retry/rename behavior for locked output files. +* `CliBuildMap` must fail fast with a clear error for locked files (non-interactive environments). + +### Agent guardrails for future changes + +* Do **not** reintroduce separate build-map logic in `Main` or other call sites. +* If map build behavior changes, update the shared `MapRequest` pipeline first, then keep wrappers thin. +* Ensure CLI and LSP builds remain behaviorally aligned unless a difference is explicitly required and tested. + +--- + +## 8. Backend Parity and Lua Guardrails + +Recent fixes established additional rules for backend work. Follow these for all future changes: + +### Jass/Lua feature parity + +* New language/compiler features must be validated for **both Jass and Lua** backends. +* Behavior should be as close as possible across backends. +* If behavior differs, treat it as intentional only when: + * the reason is backend/runtime-specific, and + * the difference is documented in tests. + +### Error behavior parity expectations + +* Prefer matching Jass behavior semantically in Lua output. +* Be explicit that Lua is stricter in some runtime cases where Jass may silently default/swallow invalid operations. +* Do not rely on Lua strictness as a substitute for correct lowering/translation. + +### Lua inliner safety: callback/function-reference boundaries + +* On Lua target, do **not** inline across callback/function-reference-heavy sites (IM `ImFuncRef`-containing callees). +* This avoids breaking callback context semantics (e.g. wrapper/xpcall/callback-native interactions such as force/group enum callbacks). +* This is a structural rule, not a name-based exclusion. + +### Lua locals limit fallback (>200 locals) + +* Lua has a hard local-variable limit per function. +* When a function exceeds the safe local threshold, rewrite locals to a locals-table fallback. +* Requirements for fallback correctness: + * locals-table declaration must be at function top before first use, + * rewritten accesses must target the declared table (no global fallback), + * nested block local initializations must be preserved, + * use deterministic **numeric slot indices** (`tbl[1]`, `tbl[2]`, ...) rather than string keys. + +### Regression testing requirements + +* Any backend parity fix must add/adjust regression tests in `tests.wurstscript.tests.*`. +* Include tests that check: + * generated backend output shape for the affected backend, + * no behavioral regression in the other backend when relevant, + * known fragile cases (dispatch binding, inlining boundaries, locals spilling). + +--- + +## 9. Virtual Slot Binding and Determinism (New Generics + Lua) + +Recent regressions showed that virtual-slot binding can silently degrade to base/no-op implementations in generated Lua while still compiling. Follow these rules for all related changes: + +### Root-slot correctness is mandatory + +* For FSM-style dispatch (`currentState.(...)`), each concrete subclass must bind that **same root slot** to its own most-specific implementation. +* Never accept mappings where a subclass has its own update method but the dispatched root slot still points to `NoOpState_*` (or another base implementation). +* When verifying generated Lua, always inspect both: + * the slot invoked at call-site (`FSM_*update`), and + * class table assignments for each sibling state class. + +### Override-chain integrity (wrapper/bridge cases) + +* If override wrappers/bridges are created, preserve transitive override links (`wrapper -> real override`) so deeper subclasses remain reachable during slot/name normalization. +* Avoid transformations that disconnect root methods from concrete overrides in the method union graph. + +### Deterministic Lua emission requirements + +* Lua output must be deterministic for identical input (same input -> byte-identical output in test harness). +* Any iteration over methods/supertypes/union groups used for naming or table assignment must be deterministic (stable ordering). +* If multiple candidate methods exist for the same slot in a class, selection must be deterministic and must prefer the most specific non-abstract implementation for that class. + +### Required regression tests for slot fixes + +* Add a repro with: + * `State`, `NoOpState`, `FSM`, + * multiple sibling `NoOpState` subclasses (including at least 4+ siblings), + * early constant state instantiation, + * root-slot call through `State`. +* In generated Lua assertions: + * extract the actual dispatched slot name from `FSM_*update` call-site, + * assert each concrete sibling class binds that slot to its own implementation, + * assert no sibling binds that dispatched slot to `NoOpState_*`. +* Add a compile-twice determinism assertion for the same repro input. diff --git a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/Main.java b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/Main.java index ce68adcb5..5d28f2c16 100644 --- a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/Main.java +++ b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/Main.java @@ -8,12 +8,11 @@ import de.peeeq.wurstio.gui.AboutDialog; import de.peeeq.wurstio.gui.WurstGuiImpl; import de.peeeq.wurstio.languageserver.LanguageServerStarter; -import de.peeeq.wurstio.languageserver.ProjectConfigBuilder; import de.peeeq.wurstio.languageserver.WFile; +import de.peeeq.wurstio.languageserver.requests.CliBuildMap; import de.peeeq.wurstio.map.importer.ImportFile; import de.peeeq.wurstio.mpq.MpqEditor; import de.peeeq.wurstio.mpq.MpqEditorFactory; -import de.peeeq.wurstio.utils.W3InstallationData; import de.peeeq.wurstscript.CompileTimeInfo; import de.peeeq.wurstscript.ErrorReporting; import de.peeeq.wurstscript.RunArgs; @@ -31,8 +30,7 @@ import java.lang.management.RuntimeMXBean; import java.nio.file.Path; import java.nio.file.Paths; -import java.nio.file.StandardCopyOption; -import java.util.LinkedList; +import java.util.ArrayList; import java.util.List; import java.util.Optional; @@ -110,35 +108,42 @@ public static void main(String[] args) { } try { - WurstProjectConfigData projectConfig = null; - Path buildDir = null; - Optional target = Optional.empty(); String workspaceroot = runArgs.getWorkspaceroot(); + RunArgs compileArgs = runArgs; + List mergedArgs = new ArrayList<>(asList(args)); + if (workspaceroot != null) { + WLogger.info("workspaceroot: " + workspaceroot); + List argsList = getCompileArgs(WFile.create(workspaceroot)); + WLogger.info("workspaceroot: " + (argsList == null)); + mergedArgs.addAll(argsList); + compileArgs = new RunArgs(mergedArgs); + } + if (runArgs.isBuild() && runArgs.getInputmap() != null && workspaceroot != null) { Path root = Paths.get(workspaceroot); Path inputMap = root.resolve(runArgs.getInputmap()); - projectConfig = WurstProjectConfig.INSTANCE.loadProject(root.resolve(FILE_NAME)); - + WurstProjectConfigData projectConfig = WurstProjectConfig.INSTANCE.loadProject(root.resolve(FILE_NAME)); if (java.nio.file.Files.exists(inputMap) && projectConfig != null) { - buildDir = root.resolve("_build"); - java.nio.file.Files.createDirectories(buildDir); - target = Optional.of(buildDir.resolve(projectConfig.getBuildMapData().getFileName() + ".w3x")); - java.nio.file.Files.copy(inputMap, target.get(), StandardCopyOption.REPLACE_EXISTING); - runArgs.setMapFile(target.get().toAbsolutePath().toString()); + CliBuildMap cliBuildMap = new CliBuildMap( + WFile.create(root.toFile()), + Optional.of(inputMap.toFile()), + mergedArgs, + Optional.empty(), + gui + ); + de.peeeq.wurstio.languageserver.ModelManager modelManager = + new de.peeeq.wurstio.languageserver.ModelManagerImpl(root.toFile(), new de.peeeq.wurstio.languageserver.BufferManager()); + modelManager.buildProject(); + Object result = cliBuildMap.execute(modelManager); + WLogger.info("map build success"); + System.out.println("Build succeeded. Output file: <" + result + ">"); + gui.sendProgress("Finished!"); + return; } } String mapFilePath = runArgs.getMapFile(); - RunArgs compileArgs = runArgs; - if (workspaceroot != null) { - WLogger.info("workspaceroot: " + workspaceroot); - List argList = new LinkedList<>(asList(args)); - List argsList = getCompileArgs(WFile.create(workspaceroot)); - WLogger.info("workspaceroot: " + (argsList == null)); - argList.addAll(argsList); - compileArgs = new RunArgs(argList); - } CompilationProcess compilationProcess = new CompilationProcess(gui, compileArgs); @Nullable CharSequence compiledScript; @@ -161,14 +166,6 @@ public static void main(String[] args) { if (compiledScript != null) { File scriptFile = new File("compiled.j.txt"); Files.write(compiledScript.toString().getBytes(Charsets.UTF_8), scriptFile); - - if (projectConfig != null && target.isPresent()) { - ProjectConfigBuilder.apply(projectConfig, target.get().toFile(), scriptFile, buildDir.toFile(), - runArgs, new W3InstallationData(null, Paths.get(workspaceroot).toFile(), false)); - - WLogger.info("map build success"); - System.out.println("Build succeeded. Output file: <" + target.get().toAbsolutePath() + ">"); - } } gui.sendProgress("Finished!"); diff --git a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/BuildMap.java b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/BuildMap.java index 060b0a087..62d48fa52 100644 --- a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/BuildMap.java +++ b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/BuildMap.java @@ -6,8 +6,6 @@ import de.peeeq.wurstio.languageserver.ModelManager; import de.peeeq.wurstio.languageserver.WFile; import de.peeeq.wurstio.languageserver.WurstLanguageServer; -import de.peeeq.wurstio.mpq.MpqEditor; -import de.peeeq.wurstio.mpq.MpqEditorFactory; import de.peeeq.wurstscript.WLogger; import de.peeeq.wurstscript.attributes.CompileError; import de.peeeq.wurstscript.gui.WurstGui; @@ -15,7 +13,6 @@ import java.io.File; import java.io.IOException; -import java.nio.file.Files; import java.util.List; import java.util.Optional; @@ -46,49 +43,7 @@ public Object execute(ModelManager modelManager) throws IOException { WLogger.info("buildMap " + map + " " + compileArgs); WurstGui gui = new WurstGuiImpl(workspaceRoot.getFile().getAbsolutePath()); try { - if (!map.isPresent()) { - throw new RequestFailedException(MessageType.Error, "Map is null"); - } - if (!map.get().exists()) { - throw new RequestFailedException(MessageType.Error, map.get().getAbsolutePath() + " does not exist."); - } - - MapRequest.mapLastModified = map.get().lastModified(); - MapRequest.mapPath = map.get().getAbsolutePath(); - - gui.sendProgress("Copying map"); - - // first we copy in same location to ensure validity - File buildDir = getBuildDir(); - String fileName = projectConfig.getBuildMapData().getFileName(); - File targetMapFile = new File(buildDir, fileName.isEmpty() ? projectConfig.getProjectName() + ".w3x" : fileName + ".w3x"); - targetMapFile = ensureWritableTargetFile( - targetMapFile, - "Build Map", - "The output map file is in use and cannot be replaced.\nClose Warcraft III and click Retry, choose Rename to use a temporary file name, or Cancel.", - "Build canceled because output map target is in use." - ); - CompilationResult result = compileScript(modelManager, gui, Optional.of(targetMapFile), projectConfig, buildDir, true); - - injectMapData(gui, Optional.of(targetMapFile), result); - - targetMapFile = ensureWritableTargetFile( - targetMapFile, - "Build Map", - "The output map file is still in use and cannot be replaced.\nClick Retry, choose Rename to use a temporary file name, or Cancel.", - "Build canceled because output map target is in use." - ); - Files.copy(getCachedMapFile().toPath(), targetMapFile.toPath(), java.nio.file.StandardCopyOption.REPLACE_EXISTING); - - gui.sendProgress("Finalizing map"); - - try (MpqEditor mpq = MpqEditorFactory.getEditor(Optional.of(targetMapFile))) { - if (mpq != null) { - mpq.closeWithCompression(); - } - } - - gui.sendProgress("Done."); + executeBuildMapPipeline(modelManager, gui, projectConfig); } catch (CompileError e) { WLogger.info(e); throw new RequestFailedException(MessageType.Error, "A compilation error occurred when building the map:\n" + e); diff --git a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/CliBuildMap.java b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/CliBuildMap.java new file mode 100644 index 000000000..b95964999 --- /dev/null +++ b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/CliBuildMap.java @@ -0,0 +1,73 @@ +package de.peeeq.wurstio.languageserver.requests; + +import config.WurstProjectConfig; +import config.WurstProjectConfigData; +import de.peeeq.wurstio.languageserver.ModelManager; +import de.peeeq.wurstio.languageserver.WFile; +import de.peeeq.wurstscript.WLogger; +import de.peeeq.wurstscript.attributes.CompileError; +import de.peeeq.wurstscript.gui.WurstGui; +import org.eclipse.lsp4j.MessageType; + +import java.io.File; +import java.io.IOException; +import java.nio.channels.FileChannel; +import java.nio.file.StandardOpenOption; +import java.util.List; +import java.util.Optional; + +import static de.peeeq.wurstio.languageserver.ProjectConfigBuilder.FILE_NAME; + +/** + * CLI entry point for build-map that reuses the same backend map pipeline as the language server. + */ +public class CliBuildMap extends MapRequest { + private final WurstGui gui; + + public CliBuildMap(WFile workspaceRoot, Optional map, List compileArgs, Optional wc3Path, WurstGui gui) { + super(null, map, compileArgs, workspaceRoot, wc3Path, Optional.empty()); + this.gui = gui; + } + + @Override + public Object execute(ModelManager modelManager) throws IOException { + if (modelManager.hasErrors()) { + throw new RequestFailedException(MessageType.Error, "Fix errors in your code before building a release.\n" + modelManager.getFirstErrorDescription()); + } + + WurstProjectConfigData projectConfig = WurstProjectConfig.INSTANCE.loadProject(workspaceRoot.getFile().toPath().resolve(FILE_NAME)); + if (projectConfig == null) { + throw new RequestFailedException(MessageType.Error, FILE_NAME + " file doesn't exist or is invalid. " + + "Please install your project using grill or the wurst setup tool."); + } + + WLogger.info("cli buildMap " + map + " " + compileArgs); + try { + File targetMapFile = executeBuildMapPipeline(modelManager, gui, projectConfig); + return targetMapFile.getAbsolutePath(); + } catch (CompileError e) { + WLogger.info(e); + throw new RequestFailedException(MessageType.Error, "A compilation error occurred when building the map:\n" + e); + } catch (Exception e) { + WLogger.warning("Exception occurred", e); + throw new RequestFailedException(MessageType.Error, "An exception was thrown when building the map:\n" + e); + } finally { + if (gui.getErrorCount() == 0) { + gui.sendFinished(); + } + } + } + + @Override + protected File ensureWritableBuildOutput(File targetFile, boolean isFinalWrite) { + if (!targetFile.exists()) { + return targetFile; + } + try (FileChannel ignored = FileChannel.open(targetFile.toPath(), StandardOpenOption.WRITE)) { + return targetFile; + } catch (IOException e) { + throw new RequestFailedException(MessageType.Warning, + "The output map file is in use and cannot be replaced: " + targetFile.getAbsolutePath()); + } + } +} diff --git a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/MapRequest.java b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/MapRequest.java index 434a75da8..e77d2562d 100644 --- a/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/MapRequest.java +++ b/de.peeeq.wurstscript/src/main/java/de/peeeq/wurstio/languageserver/requests/MapRequest.java @@ -670,6 +670,57 @@ protected void injectMapData(WurstGui gui, Optional testMap, CompilationRe WLogger.info("Cached map size after injection: " + (cachedMap.length() / 1024 / 1024) + " MB"); } + protected File executeBuildMapPipeline(ModelManager modelManager, WurstGui gui, WurstProjectConfigData projectConfig) throws Exception { + if (!map.isPresent()) { + throw new RequestFailedException(MessageType.Error, "Map is null"); + } + if (!map.get().exists()) { + throw new RequestFailedException(MessageType.Error, map.get().getAbsolutePath() + " does not exist."); + } + + mapLastModified = map.get().lastModified(); + mapPath = map.get().getAbsolutePath(); + + gui.sendProgress("Copying map"); + + File buildDir = getBuildDir(); + File targetMapFile = getBuildOutputMapFile(projectConfig, buildDir); + targetMapFile = ensureWritableBuildOutput(targetMapFile, false); + + CompilationResult result = compileScript(modelManager, gui, Optional.of(targetMapFile), projectConfig, buildDir, true); + injectMapData(gui, Optional.of(targetMapFile), result); + + targetMapFile = ensureWritableBuildOutput(targetMapFile, true); + java.nio.file.Files.copy(getCachedMapFile().toPath(), targetMapFile.toPath(), java.nio.file.StandardCopyOption.REPLACE_EXISTING); + + gui.sendProgress("Finalizing map"); + try (MpqEditor mpq = MpqEditorFactory.getEditor(Optional.of(targetMapFile))) { + if (mpq != null) { + mpq.closeWithCompression(); + } + } + + gui.sendProgress("Done."); + return targetMapFile; + } + + protected File ensureWritableBuildOutput(File targetMapFile, boolean isFinalWrite) { + String lockMessage = isFinalWrite + ? "The output map file is still in use and cannot be replaced.\nClick Retry, choose Rename to use a temporary file name, or Cancel." + : "The output map file is in use and cannot be replaced.\nClose Warcraft III and click Retry, choose Rename to use a temporary file name, or Cancel."; + return ensureWritableTargetFile( + targetMapFile, + "Build Map", + lockMessage, + "Build canceled because output map target is in use." + ); + } + + private static File getBuildOutputMapFile(WurstProjectConfigData projectConfig, File buildDir) { + String fileName = projectConfig.getBuildMapData().getFileName(); + return new File(buildDir, fileName.isEmpty() ? projectConfig.getProjectName() + ".w3x" : fileName + ".w3x"); + } + private static boolean startsWith(byte[] data, byte[] prefix) { if (data.length < prefix.length) return false; for (int i = 0; i < prefix.length; i++) {