Directory with dot not recognized correctly when loading data (breaks migration from 3.x to 4.x)

[smainz]

Environment

Liquibase Version: 4.4.3 (same error with all 4.x, no error with 3.10.x and before)

Liquibase Integration & Version: any

Liquibase Extension(s) & Version: none

Database Vendor & Version: H2 (and all others)

Operating System Type & Version: Windows 10 (and all others)

Pull Request Type

• Bug fix (non-breaking change which fixes an issue.) closes Directory with dot not recognized correctly when loading data (breaks migration from 3.x to 4.x) #2056
• Breaking change (fix or feature that would cause existing functionality to change)
• Enhancement/New feature (non-breaking change which adds functionality)

Description

Same as #2057 but with different branch name (old branch name broke the automated build)

The loadData change does not find csv files when they are resolved relative to a path which contains a dot character ('.') in the last part (i.e. db-change.log/db-changelog.xml loads a data file in db-change.log/data/file.csv).
Other lookups may be affected too by this error. See #2056

Steps To Reproduce

• clone / download https://github.com/smainz/liquibase-bug-dot-in-path
• make sure maven is installed
• run mvn test
• take a look at the README.md
• run mvn test "-Dliquibase.version=3.10.3" to see the test pass with liquibase version 3.x

Actual Behavior

• Liquibase can not find the file data/file.csv in the resource directory db-change.log in test case testLiquibaseScriptWithDotInPath()
• Liquibase can find the file data/file.csv in the resource directory db-changelog in test case testLiquibaseScriptWithoutDotInPath()

```

T E S T S

Running de.dynaware.vcheck.db.liquibase.LiquibaseTest
Aug. 31, 2021 9:22:57 VORM. liquibase.database
INFORMATION: Set default schema name to PUBLIC
Aug. 31, 2021 9:22:57 VORM. liquibase.lockservice
INFORMATION: Changelog-Protokoll erfolgreich gesperrt.
Aug. 31, 2021 9:22:57 VORM. liquibase.servicelocator
INFORMATION: Cannot load service: liquibase.parser.ChangeLogParser: liquibase.parser.core.json.JsonChangeLogParser Unable to get public no-arg constructor
Aug. 31, 2021 9:22:58 VORM. liquibase.servicelocator
INFORMATION: Cannot load service: liquibase.parser.ChangeLogParser: liquibase.parser.core.yaml.YamlChangeLogParser Unable to get public no-arg constructor
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Creating database history table with name: PUBLIC.DATABASECHANGELOG
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Reading from PUBLIC.DATABASECHANGELOG
Aug. 31, 2021 9:22:58 VORM. liquibase.lockservice
INFORMATION: Successfully released change log lock
Aug. 31, 2021 9:22:58 VORM. liquibase.database
INFORMATION: Set default schema name to PUBLIC
Aug. 31, 2021 9:22:58 VORM. liquibase.lockservice
INFORMATION: Changelog-Protokoll erfolgreich gesperrt.
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Creating database history table with name: PUBLIC.DATABASECHANGELOG
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Reading from PUBLIC.DATABASECHANGELOG
Aug. 31, 2021 9:22:58 VORM. liquibase.servicelocator
INFORMATION: Cannot load service: liquibase.hub.HubService: Provider liquibase.hub.core.StandardHubService could not be instantiated
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Table kalender created
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Index kalender_idx_datum created
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: ChangeSet db-changelog/db-changelog-master.xml::Kalender::S. Mainz ran successfully in 16ms
Aug. 31, 2021 9:22:58 VORM. liquibase.statement
INFORMATION: Executing JDBC DML batch was successful. 8 operations were executed, 1 individual UPDATE events were confirmed by the database.
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: Data loaded from 'data/some_data.csv' into table 'kalender'
Aug. 31, 2021 9:22:58 VORM. liquibase.changelog
INFORMATION: ChangeSet db-changelog/db-changelog-master.xml::Import Data::S. Mainz ran successfully in 15ms
Aug. 31, 2021 9:22:58 VORM. liquibase.lockservice
INFORMATION: Successfully released change log lock
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.672 sec <<< FAILURE!
testLiquibaseScriptWithDotInPath(de.dynaware.vcheck.db.liquibase.LiquibaseTest) Time elapsed: 1.505 sec <<< ERROR!
liquibase.exception.LiquibaseException: liquibase.exception.UnexpectedLiquibaseException: File 'data/some_data.csv' not found
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:124)
at liquibase.changelog.DatabaseChangeLog.validate(DatabaseChangeLog.java:292)
at liquibase.Liquibase.lambda$update$1(Liquibase.java:231)
at liquibase.Scope.lambda$child$0(Scope.java:166)
at liquibase.Scope.child(Scope.java:175)
at liquibase.Scope.child(Scope.java:165)
at liquibase.Scope.child(Scope.java:144)
at liquibase.Liquibase.runInScope(Liquibase.java:2404)
at liquibase.Liquibase.update(Liquibase.java:211)
at liquibase.Liquibase.update(Liquibase.java:197)
at liquibase.Liquibase.update(Liquibase.java:193)
at liquibase.Liquibase.update(Liquibase.java:185)
at de.dynaware.vcheck.db.liquibase.LiquibaseTest.testLiquibaseScriptWithDotInPath(LiquibaseTest.java:41)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: liquibase.exception.UnexpectedLiquibaseException: File 'data/some_data.csv' not found
at liquibase.change.core.LoadDataChange.generateCheckSum(LoadDataChange.java:846)
at liquibase.changelog.ChangeSet.generateCheckSum(ChangeSet.java:306)
at liquibase.changelog.ChangeSet.toString(ChangeSet.java:909)
at liquibase.changelog.ChangeLogIterator.createKey(ChangeLogIterator.java:178)
at liquibase.changelog.ChangeLogIterator.alreadySaw(ChangeLogIterator.java:186)
at liquibase.changelog.ChangeLogIterator$2.lambda$run$1(ChangeLogIterator.java:95)
at liquibase.Scope.lambda$child$0(Scope.java:166)
at liquibase.Scope.child(Scope.java:175)
at liquibase.Scope.child(Scope.java:165)
at liquibase.Scope.child(Scope.java:144)
at liquibase.Scope.child(Scope.java:228)
at liquibase.changelog.ChangeLogIterator$2.run(ChangeLogIterator.java:94)
at liquibase.Scope.lambda$child$0(Scope.java:166)
at liquibase.Scope.child(Scope.java:175)
at liquibase.Scope.child(Scope.java:165)
at liquibase.Scope.child(Scope.java:144)
at liquibase.Scope.child(Scope.java:228)
at liquibase.Scope.child(Scope.java:232)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:66)
... 46 more

Results :

Tests in error:
testLiquibaseScriptWithDotInPath(de.dynaware.vcheck.db.liquibase.LiquibaseTest): liquibase.exception.UnexpectedLiquibaseException: File 'data/some_data.csv' not found

Tests run: 2, Failures: 0, Errors: 1, Skipped: 0

## Expected/Desired Behavior
Both test cases pass and the file `data/file.csv` is found even if the change log is in a directory wich contains a dot like `db-chenge.log`.

## Fast Track PR Acceptance Checklist:
* \[ ] Build is successful and all new and existing tests pass (New test passes, existing test - unrelated to this fix -fails) 
* \[x] Added [Unit Test(s)] added condition to existing test case. 
* \[ ] Added [Integration Test(s)
* \[ ] Added [Test Harness Test(s)](https://github.com/liquibase/liquibase-test-harness/pulls)
* \[ ] Documentation Updated

- - -
## Dev Handoff Notes (Internal Use)
#### Links
* Fixed Issue: [https://github.com/liquibase/liquibase/issues/2056](https://github.com/liquibase/liquibase/issues/2056)
* Test Results: [https://github.com/liquibase/liquibase/pull/2066/checks](https://github.com/liquibase/liquibase/pull/2066/checks)

#### Testing
* Steps to Reproduce: 
  * [https://github.com/liquibase/liquibase/issues/2056](https://github.com/liquibase/liquibase/issues/2056)
  * Github repo listed above

#### Dev Verification
Ran `mvn test` in the github repo with 4.6.2 to see the error, Ran with 0-SNAPSHOT from this PR and it passed

## Test Requirements (Liquibase Internal QA)
* clone / download [https://github.com/smainz/liquibase-bug-dot-in-path](https://github.com/smainz/liquibase-bug-dot-in-path|smart-link) 
* make sure that maven is installed

**Manual Tests**

**Setup:**

* Update the pom.xml file in liquibase-bug-dot-in-path to specify the Liquibase fix branch as the version for the Liquibase dependency.

_Verify that file data/file.csv is found even if the change log is in a directory wich contains a dot like db-chenge.log._

* In the liquibase-cascade-demo directory, execute command `mvn test`
* project is built successfully

_Verify that_ `update` _is successful using maven._ 

* configure pom.xml file to connect to your database
* run `mvn compile liquibase:update`
  * table `populated` is present
  * data from names.csv file is present in `populated` table
  * data from names2.csv file is present in `populated` table
  * data from inserts.sql file is present in `populated` table

_Verify that_ `update` _is successful using CLI._ 

* run `liquibase update --changelog-file lb2178-changelog.xml`
  * `update` is successful
  * table `populated` is present
  * data from names.csv file is present in `populated` table
  * data from names2.csv file is present in `populated` table
  * data from inserts.sql file is present in `populated` table

**Automated Tests**

* None required for this fix.



┆Issue is synchronized with this [Jira Bug](https://datical.atlassian.net/browse/LB-2178) by [Unito](https://www.unito.io)

[nvoxland]

I pushed the current version of master to your fork to fix the build issue.

[molivasdat]

Hi @smainz Thanks for providing this fix

A member of the Liquibase team will take a look at your contribution and may suggest:

• Changes based on guidelines for PR submission
• Additional tests (unit and/or integration) and/or Test Harness Tests
• Provide other feedback, like requesting changes.

The PR will be prioritized according to our internal development and testing capacity.

We’ll let you know when it’s ready to move to the next step or if any changes are needed.

[smainz]

Is there anything I can help to get this PR reviewed/approved?

[nvoxland]

No, nothing more to do. I really appreciate the repo with the example project.

[sync-by-unito]

➤ Erzsebet Carmean commented:

Jack Odzhubeiskyi, whenever I test a bug that includes references to loading data, I exercise four different change types:

• loadData
• loadUpdateData
• sqlFile
• insert

In the case of this bug, the issue is with the path to the data files. Insert does not need to be tested because there is no file or path reference in the insert change type. However, it is good to keep in mind that data can be directly inserted into an existing table.

I suggest enhancing the changelog provided by the issue reporter to include loadUpdateData and sqlFile (loadData already exists).

I’ll attach a changelog and data files to illustrate what I mean.

CC Kristyl Gomes

[sync-by-unito]

➤ Erzsebet Carmean commented:

Jack Odzhubeiskyi, hi -

Could you please update the Test Requirements to have expected results for loadData, loadUpdateData and sqlFile change sets? Right now, the test case validation includes only loadData. The loadData changetype is the one that exhibited the bug, but it is a good idea to make sure related changetypes are working as expected after the fix.

I suggest that you ask Nathan Voxland if the code being changed is shared between the CLI and the Liqibase Maven integration. In general, if the code is shared, then you can do the majority of testing with the CLI. This PR came with an example Maven project and discussion with Nathan will help you understand if you should test both the CLI and Maven.

The goal of conditioning is to get enough information to craft the fewest number of test scenarios that cover the widest amount of use cases. The outcome of conditioning are test cases clear enough that anyone else on the team knows exactly what to do and exactly what to expect when they run a test. Conditioning is a skill best learned by going through the process… which you are doing! Thank you for the excellent work, Jack.