Medplum Cluster Upgrade Process #2778
codyebberson
started this conversation in
General
Replies: 2 comments 1 reply
-
Great writeup! How would data migrations work on multi-tenant instances? Would all projects have to be upgraded at the same time, or could projects opt-in individually. If the former, we should probably define some level of contract about what will/won't be considered a "data migration". For example, could the content for core FHIR resources be modified, or only content related to Medplum defined resources? |
Beta Was this translation helpful? Give feedback.
1 reply
-
Breaking this down into actionable next steps:
Tracking tickets: |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Medplum Cluster Upgrade Process
tl;dr - what you need to know as a super admin
At present, upgrading a Medplum cluster is somewhat adhoc. You probably run
medplum aws update-server
to get new features, but not much beyond that. You might be aware of the Super Admin guide, but probably only touch it if something is broken.There are a few big projects on the horizon that will be more involved. Some of these will require migrations that may take upwards of hours to complete. It is our goal to make this as simple, straightforward, and reliable as possible. (See "Examples of upcoming data migrations" below)
The main point of this discussion is the design of a new CLI command
medplum aws update
, which will orchestrate CDK upgrades, schema upgrades, and data migrations.Update 2023-12-13
This work is underway.
y
inx.y.z
)medplum aws update
is in beta testing. It retrieves the list of necessary version deployments, iteratively deploys and runs data migrations.Operating Principles
Definitions
Current State
a. Use Medplum CLI to update server and app, but very simple
a. AWS CDK can update infra, but we have not trained users to do it regularly
Proposal
a. "Schema migration" - run raw SQL to alter table definitions, should happen fast (under 1 min)
b. "Data migrations" - use
Repository
to update data or reindex data, can be slow (~hours)a. At present, "minor" version bumps do not have any particular meaning in Medplum
b. "patch" version bumps are common, usually new features and bug fixes, and do not have any migration requirements
c. "major" version bumps are rare
d. "minor" version bumps are now intended to represent a release that requires server maintenance
a. New command
medplum aws update [version]
b. Replaces
medplum aws update-server
andmedplum aws update-app
c. Determines sequence of intermediate versions required to fully upgrade the cluster
d. For each required version:
1. Deploy necessary CDK upgrades (
cdk deploy
)2. Deploy server versions (which automatically run schema migrations) (
medplum aws update-server vX.Y.Z
)3. Run data migrations
e. Can run in a loop to execute the specific sequence that is required
a. If CDK not upgraded to required version, then server will not start
b. If data migrations not run, then server will not start
Data Migrations
packages/server/src
/admin/super/migrate
POST
s to that endpoint, it will create a newAsyncJob
that runs all pending data migrationsDatabaseMigration
tablea. UPDATE 2023-12-13 - This is now managed in
AsyncJob
resourcesExamples of upcoming data migrations
a. Need to move all "R4" resources into the new "R4" project
b. Requires updating all StructureDefinition, SearchParameter, ValueSet, and CodeSystem resources
a. Need to add new
v5uuid
column to all resource tablesb. Need to compute, set, and index
a. Need to update
ValueSetElements
to be project-awarea. Need to add new
X_Reference
tables for all resource typesb. Need to reindex all resources with references (basically everything)
Beta Was this translation helpful? Give feedback.
All reactions