Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add configureReproducible( Date lastModifiedDate ) API #121

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
41 changes: 41 additions & 0 deletions src/main/java/org/codehaus/plexus/archiver/AbstractArchiver.java
Expand Up @@ -1257,4 +1257,45 @@ public String getOverrideGroupName()
{
return overrideGroupName;
}

@Override
public void configureReproducible( Date lastModifiedDate )
{
// 1. force last modified date
setLastModifiedDate( normalizeLastModifiedDate( lastModifiedDate ) );

// 2. sort filenames in each directory when scanning filesystem
setFilenameComparator( new Comparator<String>()
{
@Override
public int compare( String s1, String s2 )
{
return s1.compareTo( s2 );
}
} );

// 3. ignore file/directory mode from filesystem, since they may vary based on local user umask
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be quite inconvenient if the archive contains executable scripts (not uncommon for Java applications). I think that at least it should be explicitly stated in the configureReproducible java docs - "reproducible entries Unix mode" is not specific enough. Also I was thinking if it would make sense to keep the executable flag. Having a umask with the executable flag set to true on Unix system is quite uncommon.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, executable bit is the most wanted one.
Not sure this bit is really managed when running on Windows.
I don't know what precisely to do
I'll merge the current state as it is a good initial step: please help by providing a PR to improve

// notice: this overrides execute bit on Unix (that is already ignored on Windows)
setFileMode( Archiver.DEFAULT_FILE_MODE );
setDirectoryMode( Archiver.DEFAULT_DIR_MODE );

// 4. ignore uid/gid from filesystem (for tar)
setOverrideUid( 0 );
setOverrideUserName( "root" ); // is it possible to avoid this, like "tar --numeric-owner"?
setOverrideGid( 0 );
setOverrideGroupName( "root" );
}

/**
* Normalize last modified time value to get reproducible archive entries, based on
* archive binary format (tar uses UTC timestamp but zip uses local time then requires
* tweaks to make the value reproducible whatever the current timezone is).
*
* @param lastModifiedDate
* @return
*/
protected Date normalizeLastModifiedDate( Date lastModifiedDate )
{
return lastModifiedDate;
}
}
17 changes: 17 additions & 0 deletions src/main/java/org/codehaus/plexus/archiver/Archiver.java
Expand Up @@ -415,6 +415,9 @@ ResourceIterator getResources()
Date getLastModifiedDate();

/**
* Set filename comparator, used to sort file entries when scanning directories since File.list() does not
* guarantee any order.
*
* @since 4.2.0
*/
void setFilenameComparator( Comparator<String> filenameComparator );
Expand Down Expand Up @@ -458,4 +461,18 @@ ResourceIterator getResources()
* @since 4.2.0
*/
String getOverrideGroupName();

/**
* Configure the archiver to create archives in a reproducible way (see <a
* href="https://reproducible-builds.org/>Reproducible Builds</a>). This will configure:
* <ul>
* <li>reproducible archive entries order,</li>
* <li>defined entries timestamp</li>
* <li>and reproducible entries Unix mode.</li>
* <ul>
*
* @param lastModifiedDate the date to use for archive entries last modified time
* @since 4.2.0
*/
void configureReproducible( Date lastModifiedDate );
}
Expand Up @@ -398,4 +398,11 @@ public String getOverrideGroupName()
{
return target.getOverrideGroupName();
}

@Override
public void configureReproducible( Date lastModifiedDate )
{
target.configureReproducible( lastModifiedDate );
}

}
Expand Up @@ -414,4 +414,10 @@ public String getOverrideGroupName()
return null;
}

@Override
public void configureReproducible( Date lastModifiedDate )
{

hboutemy marked this conversation as resolved.
Show resolved Hide resolved
}

}
Expand Up @@ -468,4 +468,9 @@ public String getOverrideGroupName()
{
return null;
}

@Override
public void configureReproducible( Date lastModifiedDate )
{
}
}
Expand Up @@ -24,6 +24,8 @@
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.nio.charset.Charset;
import java.util.Calendar;
import java.util.Date;
import java.util.Hashtable;
import java.util.Stack;
import java.util.concurrent.ExecutionException;
Expand Down Expand Up @@ -833,4 +835,24 @@ protected String getArchiveType()
return archiveType;
}

@Override
protected Date normalizeLastModifiedDate( Date lastModifiedDate )
{
// timestamp of zip entries at zip storage level ignores timezone: managed in ZipEntry.setTime,
// that turns javaToDosTime: need to revert the operation here to get reproducible
// zip entry time
return new Date( dosToJavaTime( lastModifiedDate.getTime() ) );
}

/**
* Converts DOS time to Java time (number of milliseconds since epoch).
*
* @see java.util.zip.ZipEntry#setTime
* @see java.util.zip.ZipUtils#dosToJavaTime
*/
private static long dosToJavaTime( long dosTime )
{
Calendar cal = Calendar.getInstance();
return dosTime - ( cal.get( Calendar.ZONE_OFFSET ) + cal.get( Calendar.DST_OFFSET ) );
}
}