Skip to content

Commit

Permalink
Clean up contents from previously-sb tarball post binary-cleanup
Browse files Browse the repository at this point in the history
If we extracted the previously-source-build tarball only for running the
binary cleanup tool, clean up the extracted contents. This undo's the
extraction operation that was only performed for running the
binary-cleanup tool.

I am creating a tarball for building .NET offline post-prep. In that
scenario, I run ./prep-source-build.sh --bootstrap, and then package up
the entire VMR directory into a tarball. Without this cleanup, there are
a ton of duplicated binaries in the tarball, since the nuget packages
are both in previously-source-built archive and in an extracted form.
With this change, the archive size goes from just over 4GB to just over
2GB.

This is a port of dotnet/installer#19632 from
installer repo to sdk.
  • Loading branch information
omajid committed May 8, 2024
1 parent a8fc576 commit 8038487
Showing 1 changed file with 16 additions and 15 deletions.
31 changes: 16 additions & 15 deletions src/SourceBuild/content/prep-source-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -223,34 +223,35 @@ fi

if [ "$removeBinaries" == true ]; then

originalPackagesDir=$packagesDir
# Create working directory for extracking packages
workingDir=$(mktemp -d)

# If --with-packages is not passed, unpack PSB artifacts
if [[ $packagesDir == $defaultPackagesDir ]]; then
echo " Extracting previously source-built to $workingDir"
sourceBuiltArchive=$(find "$packagesArchiveDir" -maxdepth 1 -name 'Private.SourceBuilt.Artifacts*.tar.gz')

if [ ! -d "$packagesDir" ] && [ -f "$sourceBuiltArchive" ]; then
echo " Unpacking Private.SourceBuilt.Artifacts.*.tar.gz into $packagesDir"
mkdir -p "$packagesDir"
tar -xzf "$sourceBuiltArchive" -C "$packagesDir"
elif [ -f "$sourceBuiltArchive" ]; then
if [ ! -f "$packagesDir/PackageVersions.props" ]; then
echo " Creating $packagesDir/PackageVersions.props..."
tar -xzf "$sourceBuiltArchive" -C "$packagesDir" PackageVersions.props
fi
if [ ! -f "$packagesDir/VerticalManifest.xml" ]; then
echo " Unpacking Asset manifests into $packagesDir..."
tar -xzf "$sourceBuiltArchive" -C "$packagesDir" VerticalManifest.xml
fi
else
if [ ! -f "$sourceBuiltArchive" ]; then
echo " ERROR: Private.SourceBuilt.Artifacts.*.tar.gz does not exist..."\
"Cannot remove non-SB allowed binaries. Either pass --with-packages or download the artifacts."
exit 1
fi

echo " Unpacking Private.SourceBuilt.Artifacts.*.tar.gz into $workingDir"
tar -xzf "$sourceBuiltArchive" -C "$workingDir"

packagesDir=$workingDir
fi

"$REPO_ROOT/eng/detect-binaries.sh" \
"$REPO_ROOT/eng/detect-binaries.sh" \
--clean \
--allowed-binaries-file "$REPO_ROOT/eng/allowed-sb-binaries.txt" \
--with-packages $packagesDir \
--with-sdk $dotnetSdk \

rm -rf "$workingDir"

packagesDir=$originalPackagesDir
unset originalPackagesDir
fi

0 comments on commit 8038487

Please sign in to comment.