Architecture evolution

[pepicrft]

What follows is a long-term vision for aligning our architecture with the project's future while continuing to support the ecosystem with public digital goods. Hence, it challenges some past decisions and ideas that, while being key in earlier phases of the project, would significantly benefit from a shift.

Tuist started as a project generator tool and evolved into a project management tool. At its core, we continue to have a project generator, which we designed and eventually evolved to be functional and extensible to allow other features without impacting the users who would want to use the tool as a mere project generator. With the most recent decision to draw a line on the client between Tuist (open source) and Tuist Cloud features (closed-source) to ensure we could secure funding for the project, we stretched the architecture beyond its limits, leading to a bit of questionably good setup.

Moreover, talking to some users these days in Asia and thinking about how project generation can be entirely in other ecosystems like React Native or Flutter, where developers tend to be hesitant to use or maintain Xcode projects, made me think that project generation would benefit from becoming an independent component that's more discoverable and approachable.

All the above made me think about a handful of problems or needs whose ideal solution leads to re-architecting the project. Those challenges and needs are:

1. The contract between Tuist and Tuist Cloud is complicated because some areas are a bit fuzzy and implicit. Some code duplications might also lead to consistency.
2. We have three groups of models for representing the graph internals in ProjectDescription, ProjectGraph, and ProjectAutomation. This is confusing for contributors and plugin developers, who need help understanding why some graph attributes are not present in the models they are exposed to in their plugins.
3. Developers need to learn that they can add project generation at the core of their projects and, who knows, come up with interesting new ideas that wouldn't otherwise be explored. Think about the concept of "focused projects" or "binary caching" that emerged thanks to our extensibility via mappers. Project generation should be a foundation for ideas and not an exclusive component of Tuist that we are afraid to open up.

The above three challenges led me to think about the need to revisit some future architectural decisions. Note that it might be seen as unnecessary, especially when we are so settled in those models, and we can continue shipping features and fixes, but like was the case for mappers back when we introduced them, I believe this is a necessary change that will impact the project very positively.

tuist/XcodeProjectGenerator repository

I'd create a new repository, tuist/XcodeProjectGenerator, into which I'd extract the library TuistGenerator and rename it to XcodeProjectGenerator and TuistGraph, which I'd rename to XcodeProjectGraph. This last one would include the GraphTraverser utility, which helps get information from the graph. The package would follow its semantic-versioning scheme, which is feasible given the maturity of the models there, and the versioning would be decoupled from Tuist's. Note that this breaks the monorepo benefits we have experienced since the project's inception. Still, it's a sensible step forward, considering how infrequently it changes and the automation tools we can leverage to reduce our work there.

XcodeGen could be a YAML-based interface for a language-agnostic XcodeProjectGenerator.

Note

Since we are following semver, we can eliminate ProjectAutomation and iterate in our plugin system to receive models from XcodeProjectGraph instead of a subset of it. Not only that but in the very long-term, we could make the graph serializable and support server-side mappers, which would remove the need for a tuist-cloud client component because we could push to the server just the couple of mappers that need to be in the server to secure the project funding in this phase of the project.

Explicit Tuist and Tuist Cloud contract

I'd make the contract between the two very explicit and change the direction, which is only possible if we extract the common element, TuistGraph, into a different repository that both can depend on. Instead of Tuist Cloud extending Tuist, I'd make Tuist consume Tuist Cloud as just another set of features that comply with the contract established by Tuist. Not only is this easier to reason about, but it also allows other developers to follow the contract and extend it based on their needs. Developers with access to the Tuist Cloud source code could consume and integrate the sources into Tuist. The rest would pull a pre-compiled TuistCloud binary that provides that functionality. This clean contract will be very beneficial for the project, and our aim should be to eventually get rid of it, for example, by having server-side mappers.

Alternatively, we could leave things as they are. Still, as time goes by, things will continue to be more unmanageable, and we'll encounter more issues due to potential inconsistencies between the usage of mappers in Tuist and Tuist Cloud (we already have a few of those in the past).

Closing thoughts

Software is a living organism that needs to adapt as its environment changes. Sometimes, decisions that we made in the past no longer make sense in the present, and therefore, we need to challenge those with an open and future mind constantly. Moreover, we should aim to build more common goods that people can build upon because they'll improve the foundational layers, and everyone will benefit from that, including us. XcodeProj was the first one of those layers, and I believe it's time for XcodeProjectGenerator to join the family. The above changes will ensure a thriving future of Tuist Cloud and Tuist in other ecosystems that are forever stuck with Ruby and CocoaPods.

I'd love to get your thoughts on the direction. Thanks!

[kwridan]

Thanks for putting this together @pepicrft

I'm happy to see this being considered ❤️

Generally +1 for this to help solidify the generation foundation independently from the additional features Tuist offers on top of it (such as signing, caching, SPM integrations, cloud etc...). This was always my hope since #192 😅

I'll collect some additional notes on our current integration with TuistGenerator in our tooling that may help provide some feedback / datapoints for this.

[pepicrft]

I'll collect some additional notes on our current integration with TuistGenerator in our tooling that may help provide some feedback / datapoints for this.

This would be awesome. I think extracting it and treating it as an independent piece that Tuist builds upon is beneficial for Tuist because it opens it for more contributions and who knows, other tools that base themselves on the same generation foundation.