Add '-p' (preserve) option to 'cp', fixes #771 #841

dwi2 · 2018-05-03T10:54:18Z

This fixes #771.

WIP

This is still work in progress. But I want to make sure I am on the right track so I just create pull request first.

I don't plan to implement the full --preserve=ATTR_LIST feature in this PR. I think that could be a follow-up work.

TODO

Add doc and run npm run gendocs
Add test for cp -p on directory
Figure out why tests fail on Travis CI

nfischer · 2018-05-04T01:11:14Z

I kicked off a new appveyor build because the failure doesn't look related to your PR. Hopefully that's just a fluke.

dwi2 · 2018-05-04T15:05:54Z

test/cp.js

+  t.is(result.code, 0);
+
+  t.is(stat.mtime.getTime(), statOfResult.mtime.getTime());
+  t.is(stat.atime.getTime(), statOfResult.atime.getTime());


https://travis-ci.org/shelljs/shelljs/builds/374922671?utm_source=github_status&utm_medium=notification
Seems like there are some differences on atime and mtime between original file and its copy on Linux platform.
Mac OSX seems fine.
I am still figuring out why.

Probably a race. Try adding a sleep to the test (see touch tests for example), which will hopefully provide a reliable repro.

freitagbr · 2018-05-04T18:39:02Z

src/cp.js

+      fs.chownSync(destFile, srcStat.uid, srcStat.gid);
+      fs.utimesSync(destFile, srcStat.atime, srcStat.mtime);
+    }
+    fs.chmodSync(destFile, srcStat.mode);


Instead of changing the mode after writing the file, we can set the mode on the initial call to fs.openSync on line 65:

fdw = fs.openSync(destFile, 'w', srcStat.mode);

dwi2 · 2018-05-06T12:32:18Z

src/cp.js

@@ -72,10 +74,13 @@ function copyFileSync(srcFile, destFile, options) {
      pos += bytesRead;
    }

+    if (options.preserve) {
+      fs.fchownSync(fdw, srcStat.uid, srcStat.gid);
+      fs.futimesSync(fdw, srcStat.atime, srcStat.mtime);


On linux, fs.utimesSync() always produces timestamps rounded off to seconds but original files could have timestamps of millisecond level. But I still don't why is that so I probably need more time to figure it out.
However fs.futimesSync() could produce timestamps of millisecond level. So I put it this way for now.

Also I tried to add sleep in test but it didn't work.

Huh, ok. This implementation is fine. I will add a comment explaining that utimesSync doesn't work but futimesSync does.

dwi2 · 2018-05-07T13:40:55Z

Also about the appveyor build failed, I checked the difference between the build result between this PR and master branch.

In the [build of this PR]((https://ci.appveyor.com/project/shelljs/shelljs/build/1610/job/dgdr1ws2ote8fu9v), appveyor tried to install npm v6.0 on node JS v5 while it installed npm v5.8 in master build.
I couldn't find the support version of npm v6 but I suspect this is the reason why build fails. I think I also need to update appveyor.xml in this PR too.

nfischer · 2018-05-07T23:12:14Z

I think I also need to update appveyor.xml in this PR too.

Not your responsibility. I filed #844, I'll handle this ASAP.

Also I tried to add sleep in test but it didn't work.

Sleeping is supposed to break flaky tests. The suspicion is that there's a race condition, so you should add the sleep for all platforms (especially those which appear to pass). I didn't look at test output though, but examining that should make it obvious if this was because of a race.

nfischer · 2018-05-10T00:04:44Z

I think I also need to update appveyor.xml in this PR too.

Not your responsibility. I filed #844, I'll handle this ASAP.

@dwi2 FYI I fixed #844. Rebase at your convenience.

dwi2 · 2018-05-10T01:43:16Z

@nfischer Thanks a lot.

…ace condition

nfischer

Thanks for your help! I'll land this and upload a follow-up PR to address my remaining comments.

nfischer · 2018-05-10T06:39:37Z

src/cp.js

@@ -72,10 +74,13 @@ function copyFileSync(srcFile, destFile, options) {
      pos += bytesRead;
    }

+    if (options.preserve) {
+      fs.fchownSync(fdw, srcStat.uid, srcStat.gid);
+      fs.futimesSync(fdw, srcStat.atime, srcStat.mtime);


Huh, ok. This implementation is fine. I will add a comment explaining that utimesSync doesn't work but futimesSync does.

nfischer · 2018-05-10T06:50:21Z

test/cp.js

@@ -756,3 +756,29 @@ test('should not overwrite recently created files (not give error no-force mode)
  // Ensure First file is copied
  t.is(shell.cat(`${t.context.tmp}/file1`).toString(), 'test1');
 });
+
+test('cp -p should preserve mode, ownership, and timestamp', t => {
+  const result = shell.cp('-p', 'test/resources/file1', `${t.context.tmp}/preservedFile1`);


I'll follow-up to this to touch() the original file. Otherwise, we might have a race where the test setup creates file1 and the test nearly-immediately copies it--depending on the timing/precision, a regular cp() call might even pass this test.

We should probably also chmod the file too, to double-check the mode gets copied correctly (and it isn't the default mode).

nfischer · 2018-07-04T04:40:56Z

Close in favor of #869. Thanks for your help!

dwi2 · 2018-07-05T05:52:38Z

Thanks for carrying on.

dwi2 force-pushed the feat-cp-preserve branch from be4a2e1 to 7c0e83e Compare May 4, 2018 14:47

dwi2 commented May 4, 2018

View reviewed changes

freitagbr reviewed May 4, 2018

View reviewed changes

dwi2 commented May 6, 2018

View reviewed changes

nfischer mentioned this pull request May 7, 2018

Appveyor installs the latest npm, instead of the latest compatible npm #844

Closed

dwi2 added 3 commits May 10, 2018 10:43

Add '-p' (preserve) option to 'cp'

bafc9a9

Address review comment and add sleep in test to workaround possible r…

d84236d

…ace condition

Update time before fd close

3ede418

dwi2 force-pushed the feat-cp-preserve branch from c35e459 to 3ede418 Compare May 10, 2018 01:44

nfischer approved these changes May 10, 2018

View reviewed changes

dwi2 force-pushed the feat-cp-preserve branch from 7a10c1b to 8594f20 Compare May 17, 2018 14:21

Use resources/cp/file1 and add sleep to workaround possible racing

4760a76

dwi2 force-pushed the feat-cp-preserve branch from 8594f20 to 4760a76 Compare May 17, 2018 14:53

nfischer self-assigned this Jun 26, 2018

nfischer mentioned this pull request Jul 4, 2018

Add preserve option to cp #869

Merged

nfischer closed this Jul 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add '-p' (preserve) option to 'cp', fixes #771 #841

Add '-p' (preserve) option to 'cp', fixes #771 #841

dwi2 commented May 3, 2018 •

edited

nfischer commented May 4, 2018

dwi2 May 4, 2018

nfischer May 4, 2018

freitagbr May 4, 2018

dwi2 May 6, 2018 •

edited

nfischer May 10, 2018

dwi2 commented May 7, 2018

nfischer commented May 7, 2018

nfischer commented May 10, 2018

dwi2 commented May 10, 2018

nfischer left a comment

nfischer May 10, 2018

nfischer May 10, 2018

nfischer commented Jul 4, 2018

dwi2 commented Jul 5, 2018

Add '-p' (preserve) option to 'cp', fixes #771 #841

Add '-p' (preserve) option to 'cp', fixes #771 #841

Conversation

dwi2 commented May 3, 2018 • edited

WIP

TODO

nfischer commented May 4, 2018

dwi2 May 4, 2018

Choose a reason for hiding this comment

nfischer May 4, 2018

Choose a reason for hiding this comment

freitagbr May 4, 2018

Choose a reason for hiding this comment

dwi2 May 6, 2018 • edited

Choose a reason for hiding this comment

nfischer May 10, 2018

Choose a reason for hiding this comment

dwi2 commented May 7, 2018

nfischer commented May 7, 2018

nfischer commented May 10, 2018

dwi2 commented May 10, 2018

nfischer left a comment

Choose a reason for hiding this comment

nfischer May 10, 2018

Choose a reason for hiding this comment

nfischer May 10, 2018

Choose a reason for hiding this comment

nfischer commented Jul 4, 2018

dwi2 commented Jul 5, 2018

dwi2 commented May 3, 2018 •

edited

dwi2 May 6, 2018 •

edited