[RFC]: Rename Projects to Modules & Stricter Module Workflows

[HexaField]

We have a seriously capable project versioning system, that thus far has been overwhelmingly under-utilised, leading to bugs and critical errors in production.

1. Rename Project API to Package API

The Project API has grown to be far more capabale than it it was originally conceived. It began as a way of separating project code from the core engine, but now that we have many projects depending on other projects, it makes more sense to think of these as packages that augment the functionailty of the engine, that come together in ensemble to create a larger end result. Reflecting this in the terminology and documentation would go a long way to clear up ambiguities.

2. Disable packages that do not match engine version

Locally, and in production deployment, the packages API should disallow packages whose engine version does not match the current engine version. For production and local, there is no need to keep around packages that are out of date, they must either be removed or updated. This will significantly increase the incentive to keep packages up to date, as well as clean up unnecessary packages.

Having a script to update all or specific packages to the current engine version would also greatly improve the experience of large projects with multiple packages. This can also be integrated with GitHub Actions, and becomes a bit easier when #7649 is completed, such that as soon as a new version of the engine is out, the next release version becomes the current development environment.

3. Package CI/CD (see #7399) w auto-updating

A lot of this process can be automated - we have package auto-updating implemented in the engine, but without stricter versioning, it's essentially unused. If CI/CD (dev => tag => master) was tied in to package versioning, production cluster upgrades become effortless, with many more bugs and problems sorted out prior, without having to manage branches across many repositories.

4. Standardization to VSCode workspaces

As of late 2022, VSCode allows multiple repositories to have full source control capabilities, meaning engine development & project development branches can be managed entirely from within VSCode (via both the source control menu and inbuilt CLI). This may be crucial in managing multiple large projects without conflicting.

5 Core engine developer & project developer packages manifest with all relevant packages

Enforcing a workflow of including all relevant packages to a particular developer's work environment will further ensure all packages are up to date, and all projects are smoothly developed.

6. Control center integration

All of this becomes a bit more clear and organized when it has a GUI.

Project/Package API 2

This can be considered a place to discuss re-evaluating #6254

[speigg]

1. Rename Project API to Module API

I think project more closely describes what could be a bundle of code and/or assets, while module seems like a more code-centric terminology.

2. Disable modules that do not match engine version

3. Module CI/CD (see #7399) w auto-updating

This all sounds good to me

4. Standardization to VSCode workspaces

What are the implications of this? What needs to change?

5 Core engine developer & project developer modules manifest with all relevant modules

Great idea!

[jerknose]

1. Rename Project API to Module API

Not sure about the apis / naming, but it sounds like we may need both:

Projects / User defined assets, code, etc.
Plugins / Modules / Reusable code to extend the engine, etc.

###...

Everything else sounds good to me

[HexaField]

@speigg @jerknose I have revised the new name for projects/modules to packages, although i don't think it's super necessary on second thought, it's a unique abstraction that affords the capacity to use git repos as both projects and packages

[dikshashukla455]

greetings, Can I work on this issue? If it is still valid

[HexaField]

Thinking also about the issue of supporting multiple projects versions for multiple versions of the engine. If we adopt CalVer for the engine, we can adopt CalVer-SemVer for projects, in which essentially all information is captured in the project version, allowing multiple versioning schemes to coexist in the same package for different engine versions, especially if the engine release schedule slows down.

[HexaField]

Some more notes:

Each project should have a master & dev branch, when EE tags a new release version, all projects should run CI/CD that clones the engine and runs checks in the project to ensure it works with the latest release, then puts a PR up that version bumps the project and allows developers to ensure it has been properly updated

[speigg]

We should replace xrengine.config.ts with a json file that is easily modifiable in the studio without requiring engine rebuild. This will also allow us to easily do late-binding in the future (see #8868 )

Proposal: modules.json

{
  "systems": {
    "mypackage.DoSomethingSystem": "./systems/DoSomethingSystem.ts"
  }, 
  "components": {
    "mypackage.MyTypeScriptComponent": "./components/MyTypeScriptComponent.ts",
    "mypackage.MyBehaveGraphComponent": "./components/MyBehaveGraphComponent.json"
  },
  "materials": {
    "mypackage.GlowMaterial": "./materials/GlowMaterial.mat.ts"
  },
  "routes": {
    "editor": "./routes/editor.ts"
  },
  "imports": {
    "mypackage.MyState": "./MyState.ts"
  }
}

[HexaField]

We should replace xrengine.config.ts with a json file that is easily modifiable in the studio without requiring engine rebuild. This will also allow us to easily do late-binding in the future (see #8868 )

We still need a .ts file for the route imports, vite nor any other bundler will never allow deep dynamic imports for various reasons