Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add .cirrus.yml for FreeBSD (amd64, i386) CI/CD #2592

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
69 changes: 69 additions & 0 deletions .cirrus.yml
@@ -0,0 +1,69 @@
freebsd_instance:
image: freebsd-11-2-release-amd64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we use

 image_family: freebsd-11-3-snap

instead?


freebsd_build_task:

env:
matrix:
- node_js: 11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to add node 13, 12

nodeTxzKey: node-11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugly and unreadable name

npmTxzKey: npm-6
- node_js: 10
nodeTxzKey: node10
npmTxzKey: npm-node10
- node_js: 8
nodeTxzKey: node8
npmTxzKey: npm-node8
- node_js: 6
nodeTxzKey: node6
npmTxzKey: npm-node6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no longer there


env:
matrix:
- abi: freebsd:11:x86:64
- abi: freebsd:11:x86:32
jailName: j11i386
execPrefix: cbsd jexec jname=j11i386

env:
GH_API_TOKEN: ENCRYPTED[5e482f417304528148bb96eca8d030eacd6ab9972d482485fc4d42907283b995f658b351e0676e9493a37d815398f541]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xzyfer, it seems like we would need to login to CirrucCI with organization admin's account and encrypt the personal access token using the following view:

image

This is only used for publishing artifacts. 8-)


prepare_script:
- sed -i '' 's/quarterly/latest/g' /etc/pkg/FreeBSD.conf
- |
if test "$abi" = "freebsd:11:x86:32"; then
./scripts/configure_freebsd_ci_jail.sh $jailName $CIRRUS_WORKING_DIR;
$execPrefix sed -i -- '' 's/quarterly/latest/g' /etc/pkg/FreeBSD.conf;
$execPrefix pkg update -f;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we doing this only for 32 bit? Maybe this is the problem with having not update repository?

$execPrefix pkg install -y python2 > /dev/null;
fi
- $execPrefix pkg install -y c-ares gmake icu libnghttp2 libuv git > /dev/null
- |
fetch http://pkg.freebsd.org/$abi/latest/All/ -o list
nodeTxz=$(sed -n "/\"$nodeTxzKey/s/.*\"\($nodeTxzKey[^\"]*\).*/\1/p" list)
npmTxz=$(sed -n "/\"$npmTxzKey/s/.*\"\($npmTxzKey[^\"]*\).*/\1/p" list)
$execPrefix pkg add http://pkg.freebsd.org/$abi/latest/All/$nodeTxz
$execPrefix pkg add http://pkg.freebsd.org/$abi/latest/All/$npmTxz
- $execPrefix node --version
- $execPrefix npm --version
- $execPrefix clang++ --version

build_script:
- |
if test "$abi" = "freebsd:11:x86:32"; then
changeDir="cd /etc/skel &&"
fi
echo "$changeDir npm install --unsafe-perm" | $execPrefix /bin/sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should skip download here to ensure the binary is always built


publish_script:
- |
if test "$CIRRUS_TAG" != ""; then
for file in `ls vendor/**/*.node`; do
parent=${file%/*};
name=${parent##*/};
fullyQualifiedName="$(pwd)/$parent/${name}_binding.node";
mv "$file" "$parent/${name}_binding.node";
echo -e "New filename\072 $fullyQualifiedName";
./scripts/upload-github-release-asset.sh github_api_token=$GH_API_TOKEN owner=sass repo=node-sass tag=$CIRRUS_TAG filename=$fullyQualifiedName;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we know it works? Should the pipeline fail if that fails? I used your key (could not be decrypted) and the publish script finished successfully.

done
fi
2 changes: 1 addition & 1 deletion appveyor.yml
Expand Up @@ -166,7 +166,7 @@
- nodejs_version: 11
GYP_MSVS_VERSION: 2015
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015

install:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spurious change

- ps: Install-Product node $env:nodejs_version $env:platform
- node --version
Expand Down
4 changes: 2 additions & 2 deletions package.json
Expand Up @@ -76,8 +76,8 @@
"eslint": "^3.4.0",
"fs-extra": "^0.30.0",
"istanbul": "^0.4.2",
"mocha": "^3.1.2",
"mocha-lcov-reporter": "^1.2.0",
"mocha": "^5.2.0",
"mocha-lcov-reporter": "^1.3.0",
"object-merge": "^2.5.1",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated change?

"read-yaml": "^1.0.0",
"rimraf": "^2.5.2",
Expand Down
99 changes: 99 additions & 0 deletions scripts/configure_freebsd_ci_jail.sh
@@ -0,0 +1,99 @@
#!/usr/bin/env sh

set -o xtrace

jailName=$1
skelDirectory=$2
cbsd_workdir=/usr/jails
jail_arch="i386"
jail_ver="11.2"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are now at a 11.3


echo "Installing build dependencies for cbsd"
pkg install -y libssh2 rsync sqlite3 git pkgconf

echo "Clone and setup cbsd"
git clone https://github.com/cbsd/cbsd.git /usr/local/cbsd --single-branch --branch v12.0.4 --depth 1

cd /usr/local/etc/rc.d
ln -sf /usr/local/cbsd/rc.d/cbsdd
mkdir -p /usr/local/libexec/bsdconfig
cd /usr/local/libexec/bsdconfig
ln -s /usr/local/cbsd/share/bsdconfig/cbsd
pw useradd cbsd -s /bin/sh -d ${cbsd_workdir} -c "cbsd user"

# determine uplink ip address
# determine uplink iface
auto_iface=$( /sbin/route -n get 0.0.0.0 |/usr/bin/awk '/interface/{print $2}' )
my_ipv4=$( /sbin/ifconfig ${auto_iface} | /usr/bin/awk '/inet [0-9]+/{print $2}' )

if [ -z "${my_ipv4}" ]; then
echo "IPv4 not detected"
exit 1
fi

echo "Writing '${jailName}' configuration file"
cat > /tmp/${jailName}.jconf << EOF
jname="${jailName}"
path="${cbsd_workdir}/${jailName}"
host_hostname="${jailName}.my.domain"
ip4_addr="${my_ipv4}"
mount_devfs="1"
allow_mount="1"
allow_devfs="1"
allow_nullfs="1"
allow_raw_sockets="1"
mount_fstab="${cbsd_workdir}/jails-fstab/fstab.${jailName}"
arch="${jail_arch}"
mkhostsfile="1"
devfs_ruleset="4"
ver="${jail_ver}"
basename=""
baserw="0"
mount_src="0"
mount_obj="0"
mount_kernel="0"
mount_ports="1"
astart="1"
data="${cbsd_workdir}/jails-data/${jailName}-data"
vnet="0"
applytpl="1"
mdsize="0"
rcconf="${cbsd_workdir}/jails-rcconf/rc.conf_${jailName}"
floatresolv="1"
exec_poststart="0"
exec_poststop=""
exec_prestart="0"
exec_prestop="0"
exec_master_poststart="0"
exec_master_poststop="0"
exec_master_prestart="0"
exec_master_prestop="0"
pkg_bootstrap="1"
interface="0"
jailskeldir="$skelDirectory"
exec_start="/bin/sh /etc/rc"
exec_stop="/bin/sh /etc/rc.shutdown"
EOF

echo "Initializing cbsd environment"
env workdir=${cbsd_workdir} /usr/local/cbsd/sudoexec/initenv /usr/local/cbsd/share/initenv.conf

echo "Writing 'FreeBSD-bases' configuration file"
cat > ${cbsd_workdir}/etc/FreeBSD-bases.conf << EOF
auto_baseupdate=0
default_obtain_base_method="extract repo"
default_obtain_base_extract_source="/usr/freebsd-dist/base.txz"
default_obtain_base_repo_sources="https://bintray.com/am11/freebsd-dist/download_file?file_path=base-${jail_ver}-${jail_arch}.txz"
EOF

echo "Creating ${jailName}"
cbsd jcreate jconf=/tmp/${jailName}.jconf inter=0
cbsd jailscp /etc/resolv.conf ${jailName}:/etc/resolv.conf

cat > ~cbsd/jails-fstab/fstab.${jailName}.local <<EOF
${skelDirectory} /etc/skel nullfs rw 0 0
EOF

cbsd jstart jname=${jailName} inter=0

echo "${jailName} created"
60 changes: 60 additions & 0 deletions scripts/upload-github-release-asset.sh
@@ -0,0 +1,60 @@
#!/usr/bin/env sh
#
# Modified version of: https://gist.github.com/stefanbuck/ce788fee19ab6eb0b4447a85fc99f447
#
#
# This script accepts the following parameters:
#
# * owner
# * repo
# * tag
# * filename
# * github_api_token
#
# Script to upload a release asset using the GitHub API v3.
#
# Example:
#
# upload-github-release-asset.sh github_api_token=TOKEN owner=stefanbuck repo=playground tag=v0.1.0 filename=./build.zip
#

set -e

# Validate settings.
[ "$TRACE" ] && set -x

for line in "$@"; do
eval "$line"
done

# Define variables.
GH_API="https://api.github.com"
GH_REPO="$GH_API/repos/$owner/$repo"
GH_RELEASES="$GH_REPO/releases"
GH_TAGS="$GH_RELEASES/tags/$tag"
AUTH="Authorization: token $github_api_token"

if test "$tag" == "LATEST"; then
GH_TAGS="$GH_REPO/releases/latest"
fi

# Validate token.
curl -o /dev/null -sH "$AUTH" $GH_REPO || { echo "Error: Invalid repo, token or network issue!"; exit 1; }
Copy link
Member

@saper saper Apr 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming empty token:

curl -o /dev/null -sH "Authorization: token " https://api.github.com/repos/sass/node-sass && echo OK

This works, so no error reported


# Create a release from tag if it does not exist.
curl -s -d "{\"tag_name\":\"$tag\"}" -H "$AUTH" "$GH_RELEASES" > /dev/null

# Read asset tags.
response=$(curl -sH "$AUTH" $GH_TAGS)

# Get ID of the asset based on given filename.
eval "$(echo "$response" | grep -m 1 "id.:" | grep -w id | tr : = | tr -cd '[[:alnum:]]=')"
[ -n "$id" ] || { echo "Error: Failed to get release id for tag: $tag"; echo "$response" | awk 'length($0)<100' >&2; exit 1; }

# Upload asset
echo "Uploading asset... "

# Construct url
GH_ASSET="https://uploads.github.com/repos/$owner/$repo/releases/$id/assets?name=$(basename $filename)"

curl "$GITHUB_OAUTH_BASIC" --data-binary @"$filename" -H "$AUTH" -H "Content-Type: application/octet-stream" $GH_ASSET