Rethinking the Coordinator Zome <-> DNA Relationship

[mattyg]

Since the integrity-coordinator split, I think we have some confusion leftover in the relationship between Zomes & DNAs.

• Coordinator Zomes don't affect the "DNA Hash", but are still specified within the DNA manifest
• Coordinator Zomes can be swapped out freely, so aren't really "contained" in the DNA. So the biology metaphor of chromosomes -> DNA doesn't really apply anymore.

It feels like Coordinator Zomes aren't really "inside" a DNA.

How I see the "essence" of the architecture:

A hApp is:
[ Cell ] <- GUI

A Cell is:
[ Dna ] <- Coordinator

A DNA is:
[ Integrity Zome ], properties, network seed

So all together it looks like this:
[ [ Dna( [ Integrity Zome ], properties, network seed ) ] <- Coordinator ] <- GUI

@maackle's personal drawing of the architecture:

I'm afraid of this opening up a can of worms that's best left closed at this stage. But I also feel like getting this right is really important for bringing in new developers to quickly learn the framework (especially after seeing people wrestle with this confusion during the dev training).

I'm not sure if this should be left as is, and we just need better documentation and explanation. Or if we need to adjust some naming or architecture to eliminate the source of confusion.

@maackle said it best:

it is significant that this bug was likely caused by confusing naming, which led to a confused mental model (still stuck in the past where zomes are all part of DNA), which led to a confused implementation.

See this ticket for the bug that prompted this discussion: #2145

[mattyg]

One idea I had is to change the concept of a "Cell", and to add a new concept of a "LivingCell".

A DNA becomes:

• integrity zomes bundle
• properties
• network seed

A Cell becomes:

• DNA
• default coordinator zomes bundle

A LivingCell becomes:

• Cell
• AgentPubKey

But that feels a bit off as well, IDK.

[jost-s]

Composability of coordinators and integrity seems possible at first sight. However decoupling coordinators from DNA properties which are part of the DNA will not allow for liberal composition, I imagine. The DNA properties are used to store app specific values which can be queried from the UI as much as from zomes. Would that not pose a limitation to the composability?

[mattyg]

The DNA properties are used to store app specific values which can be queried from the UI as much as from zomes. Would that not pose a limitation to the composability?

Directly getting DNA properties from a UI feels like an anti-pattern to me. I feel like a coordinator function call should always be the interface from the UI into the DNA (properties and data).

[mattyg]

Intercellular coordinators could move a lot of the function of the UI into the hApp. You could write a single coordinator for your whole hApp, which is a nice way to explicitly bind your DNAs together when you don't expect them to be used as standalone modules. It could also be a nice way for devs to write bespoke integrations/bridges between existing apps. We could just say that coordinators expose the API that UIs use: UIs would connect directly to one (or more?) coordinator(s).

This is really interesting. It seems like a single coordinator zome can have a list of integrity zomes as dependencies, or have other coordinator zomes as a dependencies.

But, it doesn't make sense to me to think of coordinators as a "bundle" which are co-dependant, since they can be independently swapped out. We should surface the constraints of that composability at the level of individual coordinators.

[dukejones]

The term "RNA", though not perfect metaphorically, has a "adjacent to DNA, less permanent" connotation.
I agree, waiting until some sort of v2 to make big breaking name changes

[nphias]

So I resonate quite deeply with the tech bio-mimicry cell biology approach. Rather than creating boring generic terms to describe data structures and modules, I am a proponent for increasing the level of biomimicry.
Deployment:

Integrity bundle:
zomes (chromasomes) code. (integrity zomes)
dna.yaml (includes HDK_hash)

Coordinator bundle:
rzomes (ribosomes) code. (coordinator zomes)
rna.yaml (compaitble HDK_hash)

Happ bundle:
all zomes wasm file
happ.yaml
Receptor.json (generated API of all rzomes, receptors in biology are the gateways for information to enter the cell membrane)
holochain client UI code

---

Cell (runtime memory of all zomes+rzomes in the conductor along with agentkey and other properties):
Tissue (the container of cells for coordination around a specific role and network seed)
Organ (the container of tissues for coordination of a greater level of collective intelligence)
Cell nucleus ( remote call signal)
Cell Golgi aparatus (DHT Space )
Cell Receptors (front end components which make zome calls and listen for signals)
Integrin molecules (intra-tissue) (holochain components which handle multiple communication sequences on behalf of the agent - see BFF in microservices architecture)
biome (collection of compatible organisms for a holochain conductor eg by HDK versions)

[jost-s]

We have a task to come up with a structure for coordinator bundles, which is closely related. It seems that the Conductor architecture must change in a greater way than just internally storing coordinators independently from integrity. It should be reflected in the API too.

What about breaking up an app bundle one step further? Right now there are DNA bundles which make up an app bundle. We could have an additional step of bundling coordinators, then bundling it with integrity etc into DNA bundle and then the app bundle.

The coordinator bundle would also serve for updating coordinators without affecting DNA.

[maackle]

One thing we know we want is for clients to be able to rely on a certain set of DNA instances being available, with a certain definite interface of functions to call to read and write to those cells' source chains. And we want this "pattern" of bundled DNA to be distributable so that many nodes can be running the same constellation of cells with similar clients. That's what we currently call "hApp".

@matthme do you agree with that assessment? If so, this new notion of intercellular coordinators could cover the first part, the definite interface over a grouping of cells. We would still need some way to distribute this pattern, which could be called something else.

[nphias]

Morphogenesis is the process where cells coordinate into a higher forms of collective intelligence.. the first level is known as "Tissue" and the level above that is "Organ" (both plant and animal biology)

[maackle]

I don't know of anything in biology which maps well onto this relationship. A tissue would be multiple cells of the same DNA, but this is multiple cells with different DNA associated together, i.e. different "species". If the Cells form an Organism over the network, then an individual conscious agent would have control over particular individual cells of several different organisms. Nature doesn't have that kind of consciousness (yet) that I know of.

[nphias]

yeah I see a bit of confusion around the use of the DNA ontology.. In biology DNA is like the template language of which anything can be derived.. chromasomes, RNA, ribosomes and the rest of the cell anatomy is derived from DNA.
I would prefer to use the term "holochain DNA" and an "expression hash" being what has been derived from a specific chromasome configuration.

However I realize others are looking through the lense of multiple organisms co-existing, so each time I write zome code.. i am creating a new organism? and the result is a microbiome/colony of mutualistic organisms / endosymbionts?

From my perspective if we want to talk about different DNA within the holochain ecosystem.. it would be different co-existing versions of the holochain / HDK (conductor compatibility)

[jost-s]

This biology analogy makes things rather more complicated than easier.

[jost-s]

There can only be one set of coordinators for a DNA at a time. On installing an app, I need to provide a DNA as well as a coordinator bundle. When the coordinators of a DNA should be updated, the existing ones are replaced, they aren't retained. So whichever set of coordinators is present in the WASM DB for a DNA is the current one.

DNA hash can still be used to identify DNA, consisting of integrity and DNA modifiers. And it can also be used to associate a set of coordinators. When you're installing the app or registering a DNA, the DNA hash is used to connect the DNA with the app and also to connect the DNA with the coordinators. Internally the Conductor tracks that connection too.

[jost-s]

Fair enough, agreed. Then DNA hash cannot serve as the key of the association with coordinators. Which ties into the original problem that gave rise to this discussion.

The obvious (or the only?) key then is CellId. That has been brought up before in this discussion. It allows for mixing and matching of cells with coordinator bundles.

[matthme]

There can only be one set of coordinators for a DNA at a time

Are you talking about the status quo or also what you think is the right thing in general? I could see use cases for having multiple coordinator zome bundles for the same integrity zome bundle. I may for example want to have multiple UI's for a given set of integrity zomes that each require different coordinator zomes. Or in a scenario where multiple Admin UI's access the same conductor, it could be valuable to have them each only care about their own [coordinator zome / UI] bundles in order to not interfere with each other.

[maackle]

Given the ensuing conversation, that line would have to be updated to:

There can only be one set of coordinators for a Cell at a time

Which, unless we allow coordinators to span multiple cells, I think is vacuously true, since two sets of coordinators for a cell could also be thought of as a single set when taken together.

[matthme]

But it might matter which coordinator zomes have been installed together. If for example there are two UI's in the Launcher for a given set of integrity zomes, one of them might have an upgrade available at some point that depends on a new set of coordinator zomes. If that upgrade were to just blindly replace existing coordinator zomes that would likely break the other UI.

[nphias]

#2167 (reply in thread)

[mattyg]

I just want to surface the slightly-uncomfortable wider social context of this.

We all know holochain has gone through a lot of iteration. Personally seeing this as an outsider, with the first time with 0.1 beta it feels like we are finally starting to see the light at the end of the tunnel. Since holochain is the core dependency of the entire ecosystem, it feels like people can finally get to work on serious planning, raising money, and building on their own applications, because they have a reasonably predictable time-frame for production-readiness.

There is still only one organization funding the development of holochain, and getting them to a point of generating revenue is critical for the sustainability of the framework and the viability of the entire ecosystem built upon it.

This could became a significant detour of core developers time and attention. The business priorities and financial constraints of Holo need to be at the forefront of any decisions about this. Perhaps it might make sense to keep it as-is for now and get holochain to production-ready 1.0 release, and then do this refactoring as part of 2.0 release?

[maackle]

Yes yes, we wouldn't want to make any large change right now. It helps to think about where we'd like to get to though, even if we don't do the work now, because some things that do need to happen in the short term even if we don't restructure anything are: updating our mental models (to be able to fix actual bugs), and probably updating the way we communicate about things so others don't get confused.

For instance, we knew we wanted to rename "Header" to something else (eventually "Action") for a year or so before we actually did it, but it was on our radar for a long time until an appropriate moment came.

[ThetaSinner]

I'm not sure I have enough context to contribute too much to this, being fairly new to Holochain, but I do have some experience with packaging so here's the way I see this.

First, some observations

1. As a producer of hApps, it should be as difficult as reasonably possible to release an invalid package
2. As a consumer, it should be as clear as reasonably possible what you have installed

In a way these are old problems because they are just about distribution. Here's my summary of some different mechanisms for installing software at the OS level.

Debian

This is probably one of the simplest approaches. Your package must have a unique name and when you produce new versions you must provide a unique and orderable version.

Dependencies are declared as part of the package so that the OS can work out what other changes need to be made to the system in order to perform upgrades of the main package

Under the hood the OS care of applying the (potentially relatively complicated) package content to the system.

Relate Debian packaging back to Holochain

One of two key things for me is that package has a unique name which is user friendly. Holochain's installed_app_id is configurable by the end user which makes it user friendly but also means that the distributor of a hApp can't rely on this value to describe an upgrade to their application. The documentation currently refers to this field as follows

The unique identifier for an installed app in this conductor.
If not specified, it will be derived from the app name in the bundle manifest.

The other key thing is the ordering of versions. I'm not attached to semver and ordering that is a topic in its own right. However, from a user point of view (observation 2) it's really not clear when looking at an installed hApp what version it is running. As an end user I would want it to be easy to communicate to a friend/colleage/anybody what version I was running and which of us might need to upgrade to the latest version to be able to communicate in the case of a mismatch. The DevHub app in the Launcher shows its version as part of the name and within the devhub/launcher the hApp versions are surfaced BUT they aren't included in the manifests that I can see (by unpacking with hc).

It is worth noting that the maintainers are recommending tooling for managing patching. I've linked one of two recommendations. Holochain already has tooling in hc but it won't currently help you with checking the integrity of an upgrade (perhaps the sandbox? but not the bundle directly). This is somewhere that we might take a cue.

Windows (MSI)

(I'm sorry 😃)

I'm not sure anybody, including Microsoft is entirely happy with the MSI system but it does have some useful parallels to what holochain is doing so I think it's useful.

An MSI installer is based around the idea of features and components. A feature is as you'd expect, it's an independently installable part of an application. The components are the 'things' that make up a feature. These may be binaries, config files or whatever else is needed for that feature to operate. A component is identified by a unique ID that the distributor must specify.

The MSI system then puts a lot of onus on the distributor to decide unique identifiers. You have a product code which roughly identifies a minor version (in semver parlance) of a product. You then have a package identifier that uniquely identifies an installer (the actual MSI file). The package identifier is the key part of the versioning and integrity mechanisms because it allows an installer to react to the current state of the system. For example, I provide a -patch-3 installer against version 1 of my application and I explicitly say that my patch-1 installer must be undone, before installing patch-3.

It's a very complicated system and it's incredibly hard to get installers right (from experience) BUT it does have the advantage of being expressive and having high integrity.

Relate MSI packaging back to Holochain

If you replace globally unqiue ids with hashes then this comes closer to what Holochain is doing. A component relates quite nicely to a DNA. Then a hApp is the installation package but currently does not have a package identifier.

The logic for dealing with upgrades moves from being the responsibility of the entity doing the install (windows/holochain) to the end application. That is, installing a patched integrity zome implies changing the DNA hash and it's up to the application what to do with the data. Which actually means they need to know what was previously installed because it's not guaranteed that everyone installs the same patches or versions of an application. This could be done in terms of DNA hashes and probably should be for integrity purposes but it would be an interesting experiement to see what the cognivitive load of coding that would be. I wonder if having a version is easier for the author to understand in this case?

Android packaging

Possibly the simplest of the 3. An Android application must have a unique application id. This is not visible to the end user but is tied to the application for its lifetime. Every publish of that application must use it and anybody wanting to fork the app to build their own version must choose a new id. Google enforces this through the Play Console which means they can also enforce ownership of app ids.

The app version is split into two parts, the version code and the version name. The version code is enforced as an monotonically increasing internal version number on publish. The version name is displayed to the user.

That's really it, everything else is packaged into the app bundle. The application itself is responsible for including any database migrations that will be required. As far as I know it's not possible to unpack assets into your app's storage space so it's just up to the application to manage any assets that it has stored in previous versions.

Relate Android packaging to Holochain

Holochain has already included the idea of a friendly name that different to the emergent version of the application with the installed_app_id. A user would be free to change this to include the version they are installing if that helps them, but there is no convention for doing this that I'm aware of.

The idea of a unique application id I really like. This is not something holochain currently has and I think it would help to tie together the concept of what a hApp is from holochain's point of view.

Tie it all together (or at least try!)

Given the example of applications installed on a conductor

1. A hApp provides DNA for image sharing (hApp 1)
2. Another hApp provides DNA for adding text to images
3. A third hApp provides only coordination with an editor UI which calls a coordinator which uses both of these DNAs to let images from hApp 1 be the input to hApp 2.

Let's also say, for completeness, that the user installs two copies of hApp 2 and hApp 3. They join two different networks with the two installations for sharing content with different communities.

Entirely contrived and I'm not sure how accurate that is against the design of holochain. But what I'm trying to do is set up a sufficiently complicated example to demonstrate problems without making it more complex than it needs to be 😄

To the end user, they now have 5 upgradable units. The original 3 applications plus two installs with different agent ids. It already feels hard to reason about what should happen during an upgrade of application 3.

By taking elements of the sources above I think something like this will work well for upgrades

• Each hApp has a publisher identifier like com.thetasinner.my-awesome-happ which the publisher doesn't change between versions. This SHOULD be changed by anybody wanting to fork the application, but this can't be enforced outside of perhaps devhub.
• The publisher packages their application with anything that we can use to order releases. This could be a version number, a timestamp, an incrementing number. A packaging timestamp is nice in some ways because it can be generated by hc while packaging but equally that means two identical bundles could be given a different 'version'. I'm keen to get some input on this, or maybe some history if this has already been discussed.
• Internally, Holochain should hash the coordinator zomes and enforce integrity when a new version of the hApp is provided. If the application claims to be getting an upgrade through its version then something about the hApp must have changed. Holochain in memory storage can either key by this hash or by the publisher id + agent id or even publisher id + installed_app_id. I'm not sure which would be better, but they should be associated and persisted somewhere either way.
• The end user can install the application under the same installed_app_id as before but the publisher identifier in combination with a ordering can be used to keep the integrity of the system. It would only be possible to upgrade an application with the same publisher id. If somebody forks and wants to use the previous data then that is an application concern.
• On installation, the hApp will be told what the old configuration was and can use that to decide an appropriate upgrade strategy.

The concept of DNA stays as it is it can be thought of of it like a Windows component that can be shared by multiple hApps/agents. The concept of a long lived identifier for a stream of upgrades of the same application can be borrowed from Android and the idea of monotonically increasing versions from all 3.

I believe this addresses the two observations I opened with because a publisher has a clear set of steps to take. It should also be possible to provide tooling for validating upgrades.
The end user doesn't have to think too hard. They see an application installation. Holochain doesn't rely on the agent id or the
installed_app_id that was used. It's only possible to upgrade an application from the same publisher stream and versions are explicitly ordered.

[maackle]

Thanks for this comprehensive view drawn from experience! Here are some of my thoughts:

Seems like the notion of "updating an app" is useful, and we don't have a precise way of defining that. I imagine there is an area in between "doing nothing" and "forking an app" which could be called "updating an app". Let's look at some ways we can change an app, in increasing order of severity:

1. Changing the UI (Holochain is upstream of this change, so from our perspective it's the same as doing nothing)
2. Changing the set of coordinators available to an app
3. Adding or removing roles from the manifest, meaning we also add/remove the associated DNAs of the currently installed apps
4. Modifying existing roles by swapping out or otherwise modifying a DNA (integrity code), including changing DNA modifiers

I think only 1 and 2 could be called "updates". 3 and 4 seem like forks of the app, at least partially, since they modify the integrity layer. Thus, if we were to use a global UID for an app, an app should only be allowed to be installed with the same GUID if there is a change of type 1 or 2; a change of type 3 or 4 would require a new GUID.

As for actually enforcing a global namespace of IDs, that's outside the scope of Holochain, but could be enforced by some hApp like DevHub or the hApp Store. Not only are there naming collisions to deal with, but nonlinear version histories. Seems like a key component of enforcement is having a registry of publishers and only allowing the original publisher to make updates, but a strong part of the Holochain ethos is that any dev can update and distribute any app without asking permission, so I'm not sure how to reconcile these things. Different specific app repositories like DevHub could have a different ethos, but as far as the Conductor goes, I don't think we can count on linear version histories from known publishers. Do you see a way of reconciling this that preserves full forkability and hackability?

So my sense is that we need to split this line of thinking into what's needed for Holochain, and what are useful strategies for specific hApp distribution systems. And if I focus on the Holochain part, it looks like:

• having a way to update an installed app, meaning to update the coordinators
• which implies having some way to specify a patch to a set of coordinators (add these, remove these, replace these with these)
• which probably implies some way of locating oneself in a nonlinear version history. For instance, instead of using (GUID, version_number) to locate previous versions, each version could just list the hash of all predecessors in order, so between two apps, you know how many common versions there are (0 means fork, > 0 means an "update"). In that case, the first hash would serve the same purpose as the GUID.
• which implies being able to hash an app manifest (already close to possible if not already).

So I wonder if we can achieve the same thing as a global namespace of publishers and apps by explicitly specifying the app's update lineage as part of the app manifest.

App distribution systems could put a layer of global namespacing and sane version numbers on top of this to make everything more comprehensible for the app user.

[ThetaSinner]

This is really useful to consider the possible scenarios that might be wanted.

In the cases of 3 and 4 I think the intention is important. So the addition of a role might mean something like 'I created new functionality for my app that lets people curate existing data into lists' but say this required adding a new role so that we're in scenario 3. It is a fork from the point of view of Holochain but I think Holochain can still help match up the publisher/user expectations.

The intention of the publisher would be that existing users upgrade to the next iteration of the application. So they document the release to say that existing users should overwrite their existing install by installing against the same installed_app_id. Holochain can check that the application came from the same publisher, if it did then assume it's an intentional iteration on the application.

The user has as many different options as they please

• Follow the advice of the publisher and replace their existing app in which case everything will just work
• Do a fresh install of the latest iteration of the app under a new installed_app_id, keep using or delete their old app, and proceed from there
• Decide that they would prefer to use a fork of the app produced by a different publisher. In this case they are forced the do a side-by-side install by providing a new installed_app_id. It's still a fork but this leaves the app from the original publisher intact and visible to the end user. It's then an intentional decision by the user to either stop using and remove the old app or possibly keep it take an update for it from the original publisher in the future.

I think the 4th scenario when the integrity layer is being modified is similar but the publisher has an extra option. If the changes are significant enough to warrant releasing a different application then they might decide to update the publisher ID to make it explicit that it's a fork. So perhaps they go from an app which was published under com.thetasinner.my-app to com.thetasinner.my-awesome-app. So that it's visibly coming from the same publisher but it's a new 'stream' of releases. In that case Holochain would understand the publishers intention and force users to install side-by-side.

Whether the application goes looking for a previous version of itself to import from is only relevant to Holochain from a permissioning point of view I think. I'd expect that to be something that is built into the app and the user grants permission for the app to import from a supported previous install (at least as a suggested convention that there is a prompt).

I don't think anything about this prohibits forking/hacking on the part of publishers or users. It's a fairly light touch approach that publishers the ability to make a suggestion about what users should do with the latest iteration of an app. It also makes it slightly harder for an end user to do something they didn't intend and have no way to recover a previous installation without knowing how to dive into Holochain data.

---

For globally unique names, I agree that Holochain can't and shouldn't try to enforce anything. Maybe documenting good practise like using your Github username, or the reverse domain name of the website you host your website about the app on would be useful to people - but no restrictions required. Within DevHub I think it would still require some thought/caution. It would be very sad to have somebody move machines, lose their agent keys and be unable to reclaim ownership of an app publishing stream.

---

What I hope I'm getting to at this point, is agreeing with you about what we need in terms of a way to describe coordinator packaging/distribution. But also I hope that I've required linear histories per publisher id. There may be gaps because users have installed updates at different iterations but linear regardless. In a given installation, every time there was a branch the user would have installed side-by-side. So for example you might have com.yyy.b installed which was a fork of com.xxx.a and now you want to install com.zzz.c which is a fork of com.yyy.b. The data in com.yyy.b might be subtly different depending on whether it was created by com.yyy.b or imported from com.xxx.a but as long as the author of com.zzz.c is careful to only code against the contract of com.yyy.b they will be able to support that correctly.

I really like the idea of listing previous supported hashes. I still think it would be really nice if we could think of a way to associate those with a human readable version for feedback but hashes are much harder to make mistakes with. So your fork might declare the supported publisher id(s) and application hashes it can import data from when it gets installed.

Having version numbers at that point is where I think it could be useful for feedback purposes like 'You have version 2.3 of this application installed, I know how to import from [2.4,3) - if you want to import data please upgrade the original app'. I really don't have a clear picture of how this would be done though, because if it's asking the publisher to list 'upstream' versions against hashes then that feels onorous.

That's a whole lot of thinking out-'type' but hopefully I made some sense somewhere along. I certainly feel like I have a better idea what I mean 😄