[RFC 0672] A declarative approach to Xcode projects

[cipolleschi]

Manually handling Xcodeproj files is complicated, frustrating and anachronistic. It is also one of the main source of frustrations for React Native upgrades.

There are several tools to describe declaratively an Xcodeproj file. The one that is becoming the industry standard is Tuist.io.

This allow to describe your project in Swift, with code completion and type checking.
It is largely inspired by how Swift PM works, and it is currently maintained by our partners.

View the rendered RFC

[cortinico]

Also super excited to see this!

[brentvatne]

in my opinion, this would not make upgrades meaningfully easier.

if i'm not mistaken, it would automate is copying over the diffs from upgrade-helper for ios projects, but not much else unless every library began to document how to configure the project with tuist as part of the installation steps. there would still be the problem of android, however.

ideally, upgrading the react-native library itself should only get easier over time as react-native moves towards a stable api, which would make this change just introduce some thrash in the short-term and be unnecessary in the longer term. i'd be in favor of having a guide on how to use this but hesitant to introduce it into a template.

i believe that the hard part of upgrading isn't copying over the diffs from upgrade-helper, it's ensuring that all third party libraries in the ecosystem are up to date and compatible with the current version, and following any upgrade steps that each of those libraries has. i don't think tuist solves that. that has to come from a stable react-native api and a mature ecosystem

[pepicrft]

I would avoid requiring users to have Tuist installed because it adds unnecessary friction. Installing it as part of an NPM package package addresses the issue.

[cipolleschi]

As pointed out above, if Tuist would like to maintain that NPM package, it would be great. Otherwise, the suggested approach is a way not to depend on anyone. 😄

[pepicrft]

Once this lands, I'd like to explore removing the dependency with CocoaPods and Ruby. Developers would only need to have Node and a package manager like NPM in their environments. Solving that problem would require:

• Including a Project.swift in every React Native dependency.
• Adjusting the Tuist graph resolution to look up packages under node_modules.

We can even go further and support defining the project in the package.json:

{
  "tuist": {
    "project": {
       "targets": []
     }
  }
}

[cipolleschi]

That's not possible at the moment: we use cocoapods to install some local dependencies shipped by React Native, to run some Codegen and to configure the workspace for the user.
All the libraries in the ecosystem are also leveraging Cocoapods unfortunately.

Not that we won't be able to do it ever, but:

1. It will require a lot of work on everyone's side
2. It will require and ecosystem migration (and we are already going through one right now)
3. I think that to remove Ruby and Cocoapods, the best alternative would be Swift Package Manager.

[pepicrft]

I'm very excited to seeing this proposal. One think I'll put out there is that Tuist can be used as a tool, as suggested here, but also as a platform upon which you can build your own project generation solution. While the former is convenient because can use an existing binary and interface with the tool as any other user, the latter would open the possibility to align with the Node ecosystem:

• Supporting the definition of the project in a package.json
• Supporting dependencies under node_modules getting rid of the dependency with CocoaPods and Ruby.

[cipolleschi]

Hi @brentvatne, thank you for your thoughts:

in my opinion, this would not make upgrades meaningfully easier
if i'm not mistaken, it would automate is copying over the diffs from upgrade-helper for ios projects, but not much else unless every library began to document how to configure the project with tuist as part of the installation steps. there would still be the problem of android, however.
ideally, upgrading the react-native library itself should only get easier over time as react-native moves towards a stable api, which would make this change just introduce some thrash in the short-term and be unnecessary in the longer term. i'd be in favor of having a guide on how to use this but hesitant to introduce it into a template.
i believe that the hard part of upgrading isn't copying over the diffs from upgrade-helper, it's ensuring that all third party libraries in the ecosystem are up to date and compatible with the current version, and following any upgrade steps that each of those libraries has. i don't think tuist solves that. that has to come from a stable react-native api and a mature ecosystem

You are right. This proposal won't solve the upgrade problem in general. But that's not its goal.

One of the most reported problems when upgrading is related to managing the Xcodeproj files. Those project are very hard to read and they are structured using UUIDs which Xcode generates randomly, and that makes very hard to handle conflicts that can arise when working with them.
Also, they buries the settings and configurations of the projects and changes in the scripts, which users often miss in the upgrade helper.

By spelling out all these details, it would be much harder to miss them. 😄

Also, we would remove a big file from the repo which is always good.

Yes, Android and all the other problems will stay, but we will have one less issue to tackle! ;)

[cipolleschi]

@pepicrft the Tuist as a tool is intriguing. I'd love to learn more about it as it is not super clear how can achieve something similar, right now.

---

Anyway, thank you all for the suggestions! 😄

The proposal was not really ready to be reviewed yet (as you can also figure out from the huge amount of typos), but every feedback is always welcome!

[kelset]

this change just introduce some thrash in the short-term and be unnecessary in the longer term

I disagree on calling this "thrash" (which TIL is a different word then trash) - if we are inserting automation via tooling and making the developers' life easier in upgrading the Xcodeproj files, we should go for it. And I don't think it will be unnecessary in the future either.

[motiz88]

Does the CLI currently install anything (other than node_modules) on the user's machine? I think we may be underestimating the complexity of automating installs across different operating systems, package managers, shells, permission levels...

A guiding principle here IMO is that if we can't automate it reliably, we shouldn't automate it.

[cipolleschi]

Does the CLI currently install anything (other than node_modules) on the user's machine?
It currently installs bundle for Ruby.

Tuist is fairly easy to automate and also we can assume that the user is on a mac machine as you could not build iOS on other machines anyway. So this limits the configuration space sensibly! :D

That said, @pepicrft is suggesting an alternative way (which IIUC is basically to ship the binary in a node module, avoiding to install Tuist on the user machine) because he has the same concerns. I don't really know what this path requires, but I'm more than happy to explore it!

[lunaleaps]

Could Tuist be something we recommend vs. push on users to use? Is the intention that the upgrade-helper is simpler to follow if we adopt Tuist in the template app, but the requirement being complete adoption?

[cipolleschi]

I rewrote part of the RFC. @lunaleaps, could you check whether the new wording do answer to your questions?

There are now different proposed approach, but I think that the ideal one would be to push the tool on users that are working on App that use the CLI template and recommend to all the others to check the difference in the project description file from the upgrade helper when they have to upgrade their brownfield app or framework.

Is the intention that the upgrade-helper is simpler to follow if we adopt Tuist

That's one of the benefits

but the requirement being complete adoption?

As I said, we can go for different degrees, but I'll suggest to go for the complete adoption.

[motiz88]

I'm missing a high-level explanation of what "using Tuist" means for our users, between the motivation (why) and detailed design (how). What will be different about the experience? How will users need to adjust? Does this leave out any groups of users? (e.g. users who have some specific need to keep managing their own Xcodeproj)

[cipolleschi]

Thanks for the feedback! I will look for some time to expand this section.

Does this leave out any groups of users? (e.g. users who have some specific need to keep managing their own Xcodeproj)

To address this concern, no. What you can do while managing the Xcodeproj manually can be achieved better with Tuist. Also, nobody in the industry manages the Xcodeproj manually, because of the problem highlighted above.

[rickhanlonii]

I agree with @brentvatne. And I'll add a few thoughts.

First, I think we need to decide on the direction of the CLI template before making any meaningful changes to it like this. For example, if we intend for the CLI template to be a framework which anyone can use to create a RN app, then this change might make sense. But if the CLI template is more intended to be a reference for adding React Native to an existing app, then this change doesn't make sense because not all existing apps use tuist. So we'd need to create a second template to reference how to do it without tuist.

In my mind, we're moving towards the CLI template not being a framework, and recommending users without an app already to use a framework. That would make the CLI template a reference for native apps, and I think we should strip down dependencies as much as possible to show the "raw" RN integration. And as @brentvatne said, this focus would all us to make sure that the react-native library itself gets easier over time to upgrade and integrate without a framework and be unnecessary in the longer term.

[cipolleschi]

Hi @rickhanlonii, thanks for the feedback. Let me try to reply to it, articulating better why I think that this will be beneficial for everyone, including framework developers.

But if the CLI template is more intended to be a reference for adding React Native to an existing app, then this change doesn't make sense because not all existing apps use tuist. So we'd need to create a second template to reference how to do it without tuist.

I disagree with this statement.

The way in which a React Native application is integrated with an app depends on two aspects: code API and build logic.
Part of the build logic is defined by flags that we pass to Xcode and script that we run when a build happen and that are not represented by the more classic APIs.

Those information are now baked and buried into the Xcodeproj: in the past we had issues of people missing those bits because they were somewhere in the Xcodeproj which was not visibile.

One of the value proposition of this change is to make them more visible, using a tool (Tuist or any other, for the sake of this discussion) that is thought for that and in a language that is familiar to the developer, rather than some obsucre, Apple-specific standard.
An example of this specific request is this issue.

In my mind, we're moving towards the CLI template not being a framework, and recommending users without an app already to use a framework. That would make the CLI template a reference for native apps, and I think we should strip down dependencies as much as possible to show the "raw" RN integration.

With this statement, you are actually supporting the tool: the whole purpose of Tuist (or similar) is to strip down all the noise that the Xcodeproj introduces, making it simple and explicit for ANYONE, including framework developers, to spot the differences between versions, for what concern the build logic.

Even if users wants to keep their xcodeproj, they can always refer to the changes in these project description files to replicate them in their project.

this focus would all us to make sure that the react-native library itself gets easier over time to upgrade and integrate without a framework and be unnecessary in the longer term.

I disagree on the long term: there are companies and users that for various reasons may not be able to use a Framework, even if that is the suggested way to develop a React Native app, and I don't think we want to abandon that share of users for the foreseeable future.

[brentvatne]

@cipolleschi - it sounds like most of the proposed value here could be gained by using tuist to generate the template project, without shipping it as something that is exposed to users. when adopting to brownfield applications, folks could refer to the tuist configuration that generates the template project.

[cipolleschi]

Thank you everyone for the suggestions.

I updated and reworded the RFC using all the suggestions you provided. Specifically, I expanded the section with the benefits from adopting a similar tool, and I tried to define better the problems we are going to solve and which adoption alternatives we have.

Looking forward for a new round of feedbacks!

PS: adding another post from the community requesting something similar to Tuist.

---

@brentvatne

it sounds like most of the proposed value here could be gained by using tuist to generate the template project, without shipping it as something that is exposed to users. when adopting to brownfield applications, folks could refer to the tuist configuration that generates the template project.

For some use cases, yes. You won't need the Project.swft at all, but it will work as a much better living documentation than whatever we can write.
But I feel like that if we decide not to use it for apps created from the CLI and the template, we will lose a low hanging fruit.

[pepicrft]

I don't work on React Native actively, so take my input with a pinch of salt.

The problem with Xcode projects, even for developers that are working closer to the platform, is that some parts of it might be too intricate and hard to reason about. And the larger the project becomes, the more apparent the issue becomes.

Android's build system, Gradle, mitigates this issue on Android land. React Native uses it as a foundation to codify how NPM-dependencies are looked up under node_modules and integrated into the project at launch time. Developers just open the project and Gradle "magically" leaves them with a ready to compile project.

With the lack of such flexibility on iOS land, developers need to have:

1. A functioning Ruby setup
2. CocoaPods
3. A valid Xcode project
4. A valid graph of dependencies that CocoaPods can integrate resulting in a working project.

The amount of indirection there is insane, and issues can arise easily:

• Different results because people are using different versions of Ruby or CocoaPods
• Installation failing because the person is trying to use the macOS version of Ruby
• The project doesn't compile after integrating the dependencies because some configuration is wrong.

What about reducing that down to the equivalent of Android's Gradle but on iOS land. There's one command that will build a module graph that includes the project and its dependencies under node_modules and gives you a project that compiles. Tuist has a lot of built-in checks to catch possible issues that might lead to compilation errors. No Ruby, no CocoaPods, no project to maintain, and the version of Tuist is pinned to a version or React Native so determinism is there.

So in my opinion using it only for the template is missing a huge opportunity to eliminate what are huge sources of frustration. We can also extend Tuist with functionality that's necessary for the above to work, like being able to resolve modules using Node's module resolution. I'm very familiar with it so I can take that on the Tuist side. We could also ship the tool as an NPM package where Node calls it using native extensions. Developers don't need to install anything in their system. It just works.

I'm not against Ruby or CocoaPods, they are amazing tools and we owe a lot to them. But if we can reduce indirection and dependencies in the setup, that'll lead to a better developer experience.

How to contribute to this React Native app

1. Clone the app
2. NPM install
3. Run npm run react-native code ios

(Make sure you have Xcode installed)

And they get npm react-native cache iOS for free to turn their React Native dependencies into binaries and share them with other environments to speed up compilation times.

[ScottRobbins]

I think this proposal, using any of the 3 options listed, would be a huge improvement. However, the goal of the CLI and upgrade helper may differ between different parts of the community.

There's a large part of the community that likely comes from web development. For some, they don't want to learn how iOS projects build, what settings are changed, what react native actually does. They just want it to work, and they want it to be as easy as possible to set up. I think they would prefer the "Fully Managed" option more.

I think I fall into a different category that originally came from native development to React Native. React native often seems mysterious and "magical", but in an uncomfortable way. I don't want React Native to try to automate everything for me. I think I fall closer to the "Not managed" approach listed in this document.

I don't like the idea of React Native requiring Tuist, but instead highly suggesting it and use it as a documentation tool and template for new projects. I also don't think I want a CLI tool to be running tuist generate for me either for that reason. Tuist is great, but there are already plenty of projects using xcodeproj files or XcodeGen, and the upgrade path will be more painful for them to convert to Tuist.

[ScottRobbins]

I'm not sure I would make the assumption users are not manually updating their Project.swift file.

They're going to need to make multiple targets for things like app extensions, customizing resource locations, including precompiled frameworks that inevitably aren't distributed via a package manager, adding extra run scripts, etc.

I think this is still a good tradeoff, but educating users on what tuist does and when it needs to be invoked will be important for users.

[cipolleschi]

Yeah, thanks for specifying that. I was meaning that we do not expect major changes on the Project.swift coming from React Native itself, so we do not expect for the user to manually update the Project.swift file at all for what concerns React Native. Of course, they can modify the Project.swift for their App specific settings!

[ScottRobbins]

Thanks for the clarification :)

[ScottRobbins]

This may just be a wording issue, but I would be a little careful deleting an xcodeproj file without making it very clear to the user what the ramifications of this is.

While I agree that xcodeproj files are not a great way to manage iOS projects, they are what users have been using for a long time. At this point, they probably don't even remember what custom settings they have applied on top of the RN base project. When that file is removed, that information will be lost, and they'll be left with build failures and confusion.

In addition, Tuist will be a little bit more strict about where files are located. Again, I see this as a benefit of the tool, but existing users have definitely thrown their files around in random places, and regenerating their project won't just result in build failures, but also possibly runtime issues having to do with where they put their resources (images, localization files, etc).

[cipolleschi]

I see the point that regenerating the project if there are custom settings could be tricky.

However, the proposal aims to remove the xocdeproj from the template (so new pojects won't need it). Users that upgrades won't see their Xcodeproj removed one day from the other - the CLI will not remove it.

Tuist offers some guides on how to migrate away from the Xcodeproj. We may add links to those guides to help users migrate away from their xcodeproj.

The ultimate goal for projects would be not to have those files as they are sources of conflicts and hard to maintain. I agree that it is a delicate process, but we will all be in a better place if we manage to achieve that.

As first step, though, we may keep shipping the Xcodeproj files alongside Tuist, and then evolve the solution starting from there. How does that sound?

[ScottRobbins]

Makes sense to me!

[ScottRobbins]

Referencing #672 (comment)

I'm curious if you would feel the same way if SwiftPM could actually be used to replace Cocoapods. Would you still want to go through the trouble of more or less making your own package manager within Tuist?

[cipolleschi]

There's a large part of the community that likely comes from web development. For some, they don't want to learn how iOS projects build, what settings are changed, what react native actually does. They just want it to work, and they want it to be as easy as possible to set up. I think they would prefer the "Fully Managed" option more.

I think I fall into a different category that originally came from native development to React Native. React native often seems mysterious and "magical", but in an uncomfortable way. I don't want React Native to try to automate everything for me. I think I fall closer to the "Not managed" approach listed in this document.

That's a very good point.
We have multiple user groups and we need to keep them into consideration, as you were saying.

So far, we always thought about web developer firsts, that's why the Fully Managed flow is the one I personally prefer. It also describes the ideal end state where the users don't have to worry about the platforms details at all.

For more advanced users, I understand that they'd rather have more control over their projects. I'm curious, why don't you want for React Native to automate everything? You feel there is not enough customisation? You don't trust the platform?
Ideally, once advanced users have migrated their project, there would be no differences, so I can't see a use case, after the transition, where we don't want to generate the projects.

I don't like the idea of React Native requiring Tuist, but instead highly suggesting it and use it as a documentation tool and template for new projects.

This will probably be the very first stage of implementing this proposal as it looks like the best trade-off for everyone, right now.

I also don't think I want a CLI tool to be running tuist generate for me either for that reason. Tuist is great, but there are already plenty of projects using xcodeproj files or XcodeGen, and the upgrade path will be more painful for them to convert to Tuist.

The proposal takes an opinionated stance, I agree and understand.

The reasons why I picked Tuist over the other tool are expressed in the proposal. I was using Xcodegen in the past and migrated to Tuist and the migration was not that painful, to be honest. Tuist follow the same approach of Swift PM, and it is closer to the native platform on which we are building our apps. So, yeah, it's opinionated and users have to migrate, but if users are already using another tool to generate their projects, they would migrate to Tuist much easier than users that are not.

I'm curious if you would feel the same way if SwiftPM could actually be used to replace Cocoapods. Would you still want to go through the trouble of more or less making your own package manager within Tuist?

My personal take on this is that, the ideal end state would be:

• For React Native -> use SwiftPM to describe its dependencies and replace Cocoapods
• For React Native Apps -> use Tuist to describe the projects.

Tuist support SwiftPM packages, so the integration would be seamless.

Having Tuist runnable from node_modules, would be a huge improvement for portability of the solution, but SwiftPM is the official standard from Apple to describe dependencies and to create libraries and, at the end of the day, React Native is an iOS library to develop iOS apps (in this context). So it is more straightforward if it will be using the official solution provided by Apple.

But nothing (a part from costs) prevent us from maintaining multiple solutions. For example, even if we would be able to move to SwiftPM tomorrow, we will still have to support Cocoapods for at least a couple of year to let our users and partners migrate smoothly.

[ScottRobbins]

For more advanced users, I understand that they'd rather have more control over their projects. I'm curious, why don't you want for React Native to automate everything? You feel there is not enough customisation? You don't trust the platform?
Ideally, once advanced users have migrated their project, there would be no differences, so I can't see a use case, after the transition, where we don't want to generate the projects.

So I'll be honest, I am confused what having Tuist runnable from node_modules means. Is the Project.swift file there? Is it generated dynamically from some CLI command? Are we just saying the pbxproj gets put under node_modules?

I think I may be misunderstanding what is proposed by Pedro. The thing I'm concerned about is whether I will have enough customization.

The reasons why I picked Tuist over the other tool are expressed in the proposal. I was using Xcodegen in the past and migrated to Tuist and the migration was not that painful, to be honest.

I don't have an opinion on which tool is better. I was expressing the same concern as here.

My personal take on this is that, the ideal end state would be:
...

Looks good to me :D

(Okay, what I really want is to be able to use SwiftPM in the future to describe iOS apps itself without xcodeproj files, but I don't expect that to be possible in the near future, if ever)

[cipolleschi]

So I'll be honest, I am confused what having Tuist runnable from node_modules means. Is the Project.swift file there? Is it generated dynamically from some CLI command? Are we just saying the pbxproj gets put under node_modules?

I think I may be misunderstanding what is proposed by Pedro. The thing I'm concerned about is whether I will have enough customization.

The TL;DR: of that approach is that we don't have to install Tuist.
React Native will have a dependency on NPM which contains a Tuist executable and we would be able to run something like npx tuist <command> (I'm not sure about the syntaxm though), without having tuist installed locally.

The Project.swift will stay there and it will be written by us, plus users could customize it as if they had Tuist installed.

Ideally, pbxproj should be generated by Tuist, but if we go for the middle-ground, it will stay where it is right now.

(Okay, what I really want is to be able to use SwiftPM in the future to describe iOS apps itself without xcodeproj files, but I don't expect that to be possible in the near future, if ever)

Yeah, I share the same hope, but it's unlikely that Apple will add support for apps in SwiftPM in the short term. That's why I'm proposing this change: the Tuist syntax is very similar to SwiftPM, so the hope is that migrating to SwiftPM, whenever that happen, will be easier.

[ScottRobbins]

The TL;DR: of that approach is that we don't have to install Tuist.
React Native will have a dependency on NPM which contains a Tuist executable and we would be able to run something like npx tuist <command> (I'm not sure about the syntaxm though), without having tuist installed locally.

Thank you, I understand now and don't have any real concerns with this.