Skip to content

Support idlist option on migrate:messages #4728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Apr 23, 2021

Conversation

marvil07
Copy link
Contributor

In the context of trying to make migrate_tools compatible with drush again, this change is bringing --idlist option for the migrate:messages command.
More context at drupalspoons/migrate_tools#118: Migrate commands are now part of Drush core.

The current approach is just walking the full set.
This can be improved by using a custom \FilterIterator class to filter
the id map, as in migrate_tools implementation.
For now, only using a simple approach to avoid introducing a new class.

The current approach is just walking the full set.
This can be improved by using a custom \FilterIterator class to filter
the id map, as in migrate_tools implementation.
For now, only using a simple approach to avoid introducing a new class.
Copy link
Member

@claudiu-cristea claudiu-cristea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contribution. Apart from the inline remarks, this also needs test coverage.

@marvil07
Copy link
Contributor Author

marvil07 commented Apr 15, 2021

@claudiu-cristea, thanks for the feedback!

Can we convert one of the IDs as strings and show how to properly quote an ID containing :. Also we can explain this is usage description.

During my testing of the feature, I think I discovered a bug, I opened #4730 about it.

I made the changes related to the closure, and the documentation.

I have also added test coverage.
It was a nice and smooth experience to go over the tests 👍
I liked the woot module migration flexibility, but it took me a bit to realize it was dynamic in nature, and that failing cases are just a known set hard-coded ids on the TestFailProcess migrate process plugin.

Next steps: use the existing filter class.

Do not try to continue processing if there is nothing to process.
Move message item preprocessing into its own method to be able to reuse
it easily.

Use the provided set of values at idlist option iterating over them, to
limit the processed rows.
More details on the related inline comment.
@marvil07
Copy link
Contributor Author

That last item was less trivial than I was expecting 😅

Long story short:

  • The filter iterator on the drush codebase does its job OK, but it sadly does not apply for message retrieval from an id map.
  • There is a way in core to retrieve messages from one source ids set, but not for multiple.
  • To work-around the problem, I am just iterating over the passed values on the CLI.

More details on the inline comment.

In the process I also fixed one related error on mainline: prevent error output if the id map is empty, e.g. before migrations has been executed; and moved the pre-processing of the message item into its own method for easier re-use.

@claudiu-cristea, this is now ready for a second review.

@claudiu-cristea claudiu-cristea dismissed their stale review April 17, 2021 12:23

Issues were fixed. Creating a new review.

Copy link
Member

@claudiu-cristea claudiu-cristea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. There are only few nits left, otherwise this is good to go.

Not entirely needed but easier to read.
Convert the following:

- id{s} into ID{s},
- sql into SQL,
- Drupal Core into Drupal core,

Fix preprocessMessageRow() docblock subject.
@marvil07
Copy link
Contributor Author

@claudiu-cristea, thanks for the feedback and the attention to detail.

I have added most of the requested changes, details in context in previous review thread, went I took a slightly different path.

@marvil07
Copy link
Contributor Author

About the in-ability to filter messages on an ID map by multiple items, I created a new Drupal core issue at #3210257: Improve migrate messages retrieval.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Copy link
Member

@claudiu-cristea claudiu-cristea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution.

@claudiu-cristea claudiu-cristea merged commit 1e8fe70 into drush-ops:10.x Apr 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants