inconsistent result when trying to echo to a file

[drkarma]

Node version (or tell us if you're using electron or some other framework):

v7.10.0

ShellJS version (the most recent version/Github branch you see the bug on):

shelljs": "^0.7.8"

Operating system:

Ubuntu 16.04

Description of the bug:

Scenario:
Getting a POST from a cloud-init "phone home" function that provides some data that I want to echo into separate files per data

Bug:
There seems to be an inconsistency, because the code below does only produce correct value in the cases where the echoed string variable is short. However if I would take one of the longer string variables e.g. 'myPubKeyRSA' and create a test script like the second example in the code that works properly.
When the execution fails to echo to the file then the echoed value spills over and is displayed in the node js console

Example ShellJS command to reproduce the error:

This script will NOT put long strings into the file but the file is created

     var myPath = 'phonebook/' + myHostname + '/';
     shell.mkdir(myPath);
     shell.exec('echo ' + myHostname + ' > ' + myPath + 'hostname.txt');
     shell.echo('Wrote: ' + myHostname + ' to file');
     shell.exec('echo ' + myInstanceID + ' > ' + myPath + 'instance_id.txt');
     shell.echo('Wrote: ' + myInstanceID + ' to file');
     shell.exec('echo ' + myFQDN + ' > ' + myPath + 'fqdn.txt');
     shell.echo('Wrote: ' + myFQDN + ' to file');
     shell.exec('echo ' + myPubKeyRSA + ' > ' + myPath + 'pub_key_rsa.txt');
     shell.echo('Wrote: ' + myPubKeyRSA + ' to file');
     shell.exec('echo ' + myPubKeyECDSA + ' > ' + myPath + 'pub_key_ecdsa.txt');
     shell.echo('Wrote: ' + myPubKeyECDSA + ' to file');
     shell.exec('echo ' + myPubKeyDSA + '  > ' + myPath + 'pub_key_dsa.txt');
     shell.echo('Wrote: ' + myPubKeyDSA + ' to file');

console output when the code above is triggered

mkdir: path already exists: phonebook/centos-cloud-init/
Wrote: centos-cloud-init to file
Wrote: iid-datasource-none to file
Wrote: centos-cloud-init to file
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfF4OIX2DIUZJyr7pNqblA49C1hCWzbKZHCSi7161FvfpHJYjeL8BQQPCyhNJKN6W39iEkHbcnqdU03jImEkQT2cnOxN5LBIc0wVYtJvT+KzVj4y3bqELnjJqJjEwXZZfmBsJz4l1pnQ8mjJ0kgcgBCZCV2A9Kg6EepRoRgMbM2mrXTuMYdp+rRqrJUn7X6HBZGLS23t5ty/5B9704kDzTj+ATwZLulbuNHsBs4galz6U+5pveYseKMtRubJjCT54jCGXuyHIy6Imtetpd2KsI9pvZGvsu2STT2eX0y8lG+RNbMQuUtnhant203vog3SkpfxRhPqO8JgmGuQ64Lpy3
Wrote: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfF4OIX2DIUZJyr7pNqblA49C1hCWzbKZHCSi7161FvfpHJYjeL8BQQPCyhNJKN6W39iEkHbcnqdU03jImEkQT2cnOxN5LBIc0wVYtJvT+KzVj4y3bqELnjJqJjEwXZZfmBsJz4l1pnQ8mjJ0kgcgBCZCV2A9Kg6EepRoRgMbM2mrXTuMYdp+rRqrJUn7X6HBZGLS23t5ty/5B9704kDzTj+ATwZLulbuNHsBs4galz6U+5pveYseKMtRubJjCT54jCGXuyHIy6Imtetpd2KsI9pvZGvsu2STT2eX0y8lG+RNbMQuUtnhant203vog3SkpfxRhPqO8JgmGuQ64Lpy3
 to file
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBErLcdBMpw7Z97s9Yd+odC1hOUCGAE704HlVuzKxE5lcCQ0yRlOG11b03lAjSzNLLcqLqm/719959cjcuM8/XpY=
Wrote: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBErLcdBMpw7Z97s9Yd+odC1hOUCGAE704HlVuzKxE5lcCQ0yRlOG11b03lAjSzNLLcqLqm/719959cjcuM8/XpY=
 to file
Wrote: N/A to file

checking file in terminal

dev@vm1000:~/mk/nodeapi cat phonebook/centos-cloud-init/pub_key_rsa.txt
dev@vm1000:~/mk/nodeapi

folderlisting

dev@vm1000:~/mk/nodeapi ll phonebook/centos-cloud-init/
total 24
drwxrwxr-x 2 sdi sdi 4096 Nov  3 15:03 ./
drwxrwxr-x 3 sdi sdi 4096 Nov  3 14:58 ../
-rw-rw-r-- 1 sdi sdi   18 Nov  3 15:03 fqdn.txt
-rw-rw-r-- 1 sdi sdi   18 Nov  3 15:03 hostname.txt
-rw-rw-r-- 1 sdi sdi   20 Nov  3 15:03 instance_id.txt
-rw-rw-r-- 1 sdi sdi    4 Nov  3 15:03 pub_key_dsa.txt
-rw-rw-r-- 1 sdi sdi    0 Nov  3 15:03 pub_key_ecdsa.txt
-rw-rw-r-- 1 sdi sdi    0 Nov  3 15:03 pub_key_rsa.txt

This script will put value into a file even if the variable holds a long string

var myKeyValue = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfF4OIX2DIUZJyr7pNqblA49C1hCWzbKZHCSi7161FvfpHJYjeL8BQQPCyhNJKN6W39iEkHbcnqdU03jImEkQT2cnOxN5LBIc0wVYtJvT+KzVj4y3bqELnjJqJjEwXZZfmBsJz4l1pnQ8mjJ0kgcgBCZCV2A9Kg6EepRoRgMbM2mrXTuMYdp+rRqrJUn7X6HBZGLS23t5ty/5B9704kDzTj+ATwZLulbuNHsBs4galz6U+5pveYseKMtRubJjCT54jCGXuyHIy6Imtetpd2KsI9pvZGvsu2STT2eX0y8lG+RNbMQuUtnhant203vog3SkpfxRhPqO8JgmGuQ64Lpy3";
var myPath = 'phonebook/';

shell.exec('echo ' + myKeyValue + ' > ' + myPath + 'thekey.txt');

Output in terminal

dev@vm1000:~/mk/nodeapi cat phonebook/thekey.txt
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfF4OIX2DIUZJyr7pNqblA49C1hCWzbKZHCSi7161FvfpHJYjeL8BQQPCyhNJKN6W39iEkHbcnqdU03jImEkQT2cnOxN5LBIc0wVYtJvT+KzVj4y3bqELnjJqJjEwXZZfmBsJz4l1pnQ8mjJ0kgcgBCZCV2A9Kg6EepRoRgMbM2mrXTuMYdp+rRqrJUn7X6HBZGLS23t5ty/5B9704kDzTj+ATwZLulbuNHsBs4galz6U+5pveYseKMtRubJjCT54jCGXuyHIy6Imtetpd2KsI9pvZGvsu2STT2eX0y8lG+RNbMQuUtnhant203vog3SkpfxRhPqO8JgmGuQ64Lpy3

[nfischer]

Edit: I wrapped some of your text in code blocks in the description

[nfischer]

I'm a bit confused (this is a lot of text). @drkarma Could you clarify:

1. Is this a bug with shell.echo(), shell.exec('echo ...'), or both?
2. What happens if you use shell.echo('my text to echo').to('myFile.txt')?

---

In general, I would recommend trying something like:

shell.config.silent = true; // Don't let echo print to the terminal
shell.echo('my test to echo').to('myFile.txt'); // This is cross-platform

[drkarma]

Hi sorry to confuse you.
The "shell.echo()" parts are there just for troubleshooting.

Doing what you state e.g. shell.echo(myPubKeyRSA).to(myPath + 'pub_key_rsa.txt'); (and I slap my forehead for not figuring that out myself) solves my problem...but the reported issue remains.

I.e. this works

var myKeyValue = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfF4OIX2DIUZJyr7pNqblA49C1hCWzbKZHCSi7161FvfpHJYjeL8BQQPCyhNJKN6W39iEkHbcnqdU03jImEkQT2cnOxN5LBIc0wVYtJvT+KzVj4y3bqELnjJqJjEwXZZfmBsJz4l1pnQ8mjJ0kgcgBCZCV2A9Kg6EepRoRgMbM2mrXTuMYdp+rRqrJUn7X6HBZGLS23t5ty/5B9704kDzTj+ATwZLulbuNHsBs4galz6U+5pveYseKMtRubJjCT54jCGXuyHIy6Imtetpd2KsI9pvZGvsu2STT2eX0y8lG+RNbMQuUtnhant203vog3SkpfxRhPqO8JgmGuQ64Lpy3";
var myPath = 'phonebook/';

shell.exec('echo ' + myKeyValue + ' > ' + myPath + 'thekey.txt');

But when the same thing done when processed in the first code snip does not.

i.e. the following

var myPath = 'phonebook/' + myHostname + '/';
     shell.mkdir(myPath);
      ... skipped code
     shell.exec('echo ' + myPubKeyRSA + ' > ' + myPath + 'pub_key_rsa.txt');
      ... skipped code

But your suggestion sorted it out for me so I am happy, but I still think it is odd that there is an inconsistency.
Regards,
Magnus

[nfischer]

But your suggestion sorted it out for me so I am happy

Great news! 🎉

---

I can't quite figure out what issue you were seeing with exec. I tried exec('echo...'), but it's echoing things as expected.

[nfischer]

Closing, because we can't reproduce (and the reporter has a workaround).