tmac build: conform to macOS 10.15 Gatekeeper requirements - electrum - Electrum Bitcoin wallet
(HTM) git clone https://git.parazyd.org/electrum
(DIR) Log
(DIR) Files
(DIR) Refs
(DIR) Submodules
---
(DIR) commit 095464b620e3e1bb699d174d969526f78ad356b9
(DIR) parent 9baaf1afda3181cd511fda83c0be426a85c02abd
(HTM) Author: SomberNight <somber.night@protonmail.com>
Date: Tue, 12 May 2020 12:37:40 +0200
mac build: conform to macOS 10.15 Gatekeeper requirements
fixes #6128
some of this is based on:
https://github.com/metabrainz/picard/blob/e1354632d2db305b7a7624282701d34d73afa225/scripts/package/macos-notarize-app.sh
https://github.com/Electron-Cash/Electron-Cash/commit/1eb8b71e7d11141432f1c46629683a5a703795e2
https://github.com/Electron-Cash/Electron-Cash/commit/24e44e9784fa23fa9f408ce3f9489fac8568093b
https://github.com/Electron-Cash/Electron-Cash/commit/5abec73eee0cdeb725e3c5a989621ec4ccfb92a0
Diffstat:
M contrib/osx/README.md | 65 ++++++++++++++++---------------
D contrib/osx/base.sh | 23 -----------------------
A contrib/osx/entitlements.plist | 19 +++++++++++++++++++
M contrib/osx/make_osx | 62 +++++++++++++++++++++++--------
A contrib/osx/notarize_app.sh | 77 +++++++++++++++++++++++++++++++
5 files changed, 175 insertions(+), 71 deletions(-)
---
(DIR) diff --git a/contrib/osx/README.md b/contrib/osx/README.md
t@@ -1,5 +1,5 @@
-Building Mac OS binaries
-========================
+Building macOS binaries
+=======================
✗ _This script does not produce reproducible output (yet!).
Please help us remedy this._
t@@ -7,36 +7,47 @@ Building Mac OS binaries
This guide explains how to build Electrum binaries for macOS systems.
-## 1. Building the binary
+## Building the binary
-This needs to be done on a system running macOS or OS X. We use El Capitan (10.11.6) as building it
-on High Sierra (or later)
-makes the binaries [incompatible with older versions](https://github.com/pyinstaller/pyinstaller/issues/1191).
+This needs to be done on a system running macOS or OS X.
-Another factor for the minimum supported macOS version is the
-[bundled Qt version](https://github.com/spesmilo/electrum/issues/3685).
+Notes about compatibility with different macOS versions:
+- In general the binary is not guaranteed to run on an older version of macOS
+ than what the build machine has. This is due to bundling the compiled Python into
+ the [PyInstaller binary](https://github.com/pyinstaller/pyinstaller/issues/1191).
+- The [bundled version of Qt](https://github.com/spesmilo/electrum/issues/3685) also
+ imposes a minimum supported macOS version.
+- If you want to build binaries that conform to the macOS "Gatekeeper", so as to
+ minimise the warnings users get, the binaries need to be codesigned with a
+ certificate issued by Apple, and starting with macOS 10.15 the binaries also
+ need to be notarized by Apple's central server. The catch is that to be able to build
+ binaries that Apple will notarise (due to the requirements on the binaries themselves,
+ e.g. hardened runtime) the build machine needs at least macOS 10.14.
+ See [#6128](https://github.com/spesmilo/electrum/issues/6128).
+
+We currently build the release binaries on macOS 10.14.6, and these seem to run on
+10.13 or newer.
Before starting, make sure that the Xcode command line tools are installed (e.g. you have `git`).
-#### 1.1a Get Xcode
+#### 1.a Get Xcode
Building the QR scanner (CalinsQRReader) requires full Xcode (not just command line tools).
-The last Xcode version compatible with El Capitan is Xcode 8.2.1
-
Get it from [here](https://developer.apple.com/download/more/).
-
Unfortunately, you need an "Apple ID" account.
+(note: the last Xcode that runs on macOS 10.14.6 is Xcode 11.3.1)
+
After downloading, uncompress it.
Make sure it is the "selected" xcode (e.g.):
sudo xcode-select -s $HOME/Downloads/Xcode.app/Contents/Developer/
-#### 1.1b Build QR scanner separately on newer Mac
+#### 1.b Build QR scanner separately on another Mac
-Alternatively, you can try building just the QR scanner on newer macOS.
+Alternatively, you can try building just the QR scanner on another Mac.
On newer Mac, run:
t@@ -46,27 +57,17 @@ On newer Mac, run:
Move `prebuilt_qr` to El Capitan: `contrib/osx/CalinsQRReader/prebuilt_qr`.
-#### 1.2 Build Electrum
+#### 2. Build Electrum
cd electrum
./contrib/osx/make_osx
-
-This creates both a folder named Electrum.app and the .dmg file.
-
-## 2. Building the image deterministically (WIP)
-The usual way to distribute macOS applications is to use image files containing the
-application. Although these images can be created on a Mac with the built-in `hdiutil`,
-they are not deterministic.
-
-Instead, we use the toolchain that Bitcoin uses: genisoimage and libdmg-hfsplus.
-These tools do not work on macOS, so you need a separate Linux machine (or VM).
-
-Copy the Electrum.app directory over and install the dependencies, e.g.:
+This creates both a folder named Electrum.app and the .dmg file.
- apt install libcap-dev cmake make gcc faketime
-
-Then you can just invoke `package.sh` with the path to the app:
+If you want the binaries codesigned for MacOS and notarised by Apple's central server,
+provide these env vars to the `make_osx` script:
- cd electrum
- ./contrib/osx/package.sh ~/Electrum.app/
+ CODESIGN_CERT="Developer ID Application: Electrum Technologies GmbH (L6P37P7P56)" \
+ APPLE_ID_USER="me@email.com" \
+ APPLE_ID_PASSWORD="1234" \
+ ./contrib/osx/make_osx
(DIR) diff --git a/contrib/osx/base.sh b/contrib/osx/base.sh
t@@ -1,23 +0,0 @@
-#!/usr/bin/env bash
-
-. $(dirname "$0")/../build_tools_util.sh
-
-
-function DoCodeSignMaybe { # ARGS: infoName fileOrDirName codesignIdentity
- infoName="$1"
- file="$2"
- identity="$3"
- deep=""
- if [ -z "$identity" ]; then
- # we are ok with them not passing anything; master script calls us unconditionally even if no identity is specified
- return
- fi
- if [ -d "$file" ]; then
- deep="--deep"
- fi
- if [ -z "$infoName" ] || [ -z "$file" ] || [ -z "$identity" ] || [ ! -e "$file" ]; then
- fail "Argument error to internal function DoCodeSignMaybe()"
- fi
- info "Code signing ${infoName}..."
- codesign -f -v $deep -s "$identity" "$file" || fail "Could not code sign ${infoName}"
-}
(DIR) diff --git a/contrib/osx/entitlements.plist b/contrib/osx/entitlements.plist
t@@ -0,0 +1,19 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+ <!-- These are required for binaries built by PyInstaller -->
+ <!-- see pyinstaller/pyinstaller#4629 -->
+ <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
+ <true/>
+ <key>com.apple.security.cs.disable-library-validation</key>
+ <true/>
+
+ <!-- These are required for USB HID access (hw wallets). -->
+ <!-- see https://github.com/Electron-Cash/Electron-Cash/commit/5abec73eee0cdeb725e3c5a989621ec4ccfb92a0 -->
+ <key>com.apple.security.cs.allow-dyld-environment-variables</key>
+ <true/>
+ <key>com.apple.security.cs.allow-jit</key>
+ <true/>
+</dict>
+</plist>
(DIR) diff --git a/contrib/osx/make_osx b/contrib/osx/make_osx
t@@ -5,11 +5,12 @@ PYTHON_VERSION=3.7.6
BUILDDIR=/tmp/electrum-build
PACKAGE=Electrum
GIT_REPO=https://github.com/spesmilo/electrum
-LIBSECP_VERSION="b408c6a8b287003d1ade5709e6f7bc3c7f1d5be7"
export GCC_STRIP_BINARIES="1"
-. $(dirname "$0")/base.sh
+
+. $(dirname "$0")/../build_tools_util.sh
+
CONTRIB_OSX="$(dirname "$(realpath "$0")")"
CONTRIB="$CONTRIB_OSX/.."
t@@ -24,26 +25,46 @@ which brew > /dev/null 2>&1 || fail "Please install brew from https://brew.sh/ t
which xcodebuild > /dev/null 2>&1 || fail "Please install Xcode and xcode command line tools to continue"
# Code Signing: See https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html
-APP_SIGN=""
-if [ -n "$1" ]; then
+if [ -n "$CODESIGN_CERT" ]; then
# Test the identity is valid for signing by doing this hack. There is no other way to do this.
cp -f /bin/ls ./CODESIGN_TEST
- codesign -s "$1" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1
+ codesign -s "$CODESIGN_CERT" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1
res=$?
rm -f ./CODESIGN_TEST
if ((res)); then
- fail "Code signing identity \"$1\" appears to be invalid."
+ fail "Code signing identity \"$CODESIGN_CERT\" appears to be invalid."
fi
unset res
- APP_SIGN="$1"
- info "Code signing enabled using identity \"$APP_SIGN\""
+ info "Code signing enabled using identity \"$CODESIGN_CERT\""
else
- warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system as the first argument to this script to enable signing."
+ warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system to enable signing."
fi
+
+function DoCodeSignMaybe { # ARGS: infoName fileOrDirName
+ infoName="$1"
+ file="$2"
+ deep=""
+ if [ -z "$CODESIGN_CERT" ]; then
+ # no cert -> we won't codesign
+ return
+ fi
+ if [ -d "$file" ]; then
+ deep="--deep"
+ fi
+ if [ -z "$infoName" ] || [ -z "$file" ] || [ ! -e "$file" ]; then
+ fail "Argument error to internal function DoCodeSignMaybe()"
+ fi
+ hardened_arg="--entitlements=${CONTRIB_OSX}/entitlements.plist -o runtime"
+
+ info "Code signing ${infoName}..."
+ codesign -f -v $deep -s "$CODESIGN_CERT" $hardened_arg "$file" || fail "Could not code sign ${infoName}"
+}
+
+
info "Installing Python $PYTHON_VERSION"
export PATH="~/.pyenv/bin:~/.pyenv/shims:~/Library/Python/3.7/bin:$PATH"
-if [ -d "~/.pyenv" ]; then
+if [ -d "${HOME}/.pyenv" ]; then
pyenv update
else
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash > /dev/null 2>&1
t@@ -109,7 +130,7 @@ rm -fr build
# prefer building using xcode ourselves. otherwise fallback to prebuilt binary
xcodebuild || cp -r prebuilt_qr build || fail "Could not build CalinsQRReader"
popd
-DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop
+DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app"
info "Installing requirements..."
t@@ -131,7 +152,7 @@ for d in ~/Library/Python/ ~/.pyenv .; do
done
info "Building binary"
-APP_SIGN="$APP_SIGN" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary"
+APP_SIGN="$CODESIGN_CERT" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary"
info "Adding bitcoin URI types to Info.plist"
plutil -insert 'CFBundleURLTypes' \
t@@ -139,14 +160,23 @@ plutil -insert 'CFBundleURLTypes' \
-- dist/$PACKAGE.app/Contents/Info.plist \
|| fail "Could not add keys to Info.plist. Make sure the program 'plutil' exists and is installed."
-DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop
+DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app"
+
+if [ ! -z "$CODESIGN_CERT" ]; then
+ if [ ! -z "$APPLE_ID_USER" ]; then
+ info "Notarizing .app with Apple's central server..."
+ "${CONTRIB_OSX}/notarize_app.sh" "dist/${PACKAGE}.app" || fail "Could not notarize binary."
+ else
+ warn "AppleID details not set! Skipping Apple notarization."
+ fi
+fi
info "Creating .DMG"
hdiutil create -fs HFS+ -volname $PACKAGE -srcfolder dist/$PACKAGE.app dist/electrum-$VERSION.dmg || fail "Could not create .DMG"
-DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg" "$APP_SIGN" # If APP_SIGN is empty will be a noop
+DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg"
-if [ -z "$APP_SIGN" ]; then
+if [ -z "$CODESIGN_CERT" ]; then
warn "App was built successfully but was not code signed. Users may get security warnings from macOS."
- warn "Specify a valid code signing identity as the first argument to this script to enable code signing."
+ warn "Specify a valid code signing identity to enable code signing."
fi
(DIR) diff --git a/contrib/osx/notarize_app.sh b/contrib/osx/notarize_app.sh
t@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+# from https://github.com/metabrainz/picard/blob/e1354632d2db305b7a7624282701d34d73afa225/scripts/package/macos-notarize-app.sh
+
+
+if [ -z "$1" ]; then
+ echo "Specify app bundle as first parameter"
+ exit 1
+fi
+
+if [ -z "$APPLE_ID_USER" ] || [ -z "$APPLE_ID_PASSWORD" ]; then
+ echo "You need to set your Apple ID credentials with \$APPLE_ID_USER and \$APPLE_ID_PASSWORD."
+ exit 1
+fi
+
+APP_BUNDLE=$(basename "$1")
+APP_BUNDLE_DIR=$(dirname "$1")
+
+cd "$APP_BUNDLE_DIR" || exit 1
+
+# Package app for submission
+echo "Generating ZIP archive ${APP_BUNDLE}.zip..."
+ditto -c -k --rsrc --keepParent "$APP_BUNDLE" "${APP_BUNDLE}.zip"
+
+# Submit for notarization
+echo "Submitting $APP_BUNDLE for notarization..."
+RESULT=$(xcrun altool --notarize-app --type osx \
+ --file "${APP_BUNDLE}.zip" \
+ --primary-bundle-id org.electrum.electrum \
+ --username $APPLE_ID_USER \
+ --password @env:APPLE_ID_PASSWORD \
+ --output-format xml)
+
+if [ $? -ne 0 ]; then
+ echo "Submitting $APP_BUNDLE failed:"
+ echo "$RESULT"
+ exit 1
+fi
+
+REQUEST_UUID=$(echo "$RESULT" | xpath \
+ "//key[normalize-space(text()) = 'RequestUUID']/following-sibling::string[1]/text()" 2> /dev/null)
+
+if [ -z "$REQUEST_UUID" ]; then
+ echo "Submitting $APP_BUNDLE failed:"
+ echo "$RESULT"
+ exit 1
+fi
+
+echo "$(echo "$RESULT" | xpath \
+ "//key[normalize-space(text()) = 'success-message']/following-sibling::string[1]/text()" 2> /dev/null)"
+
+# Poll for notarization status
+echo "Submitted notarization request $REQUEST_UUID, waiting for response..."
+sleep 60
+while :
+do
+ RESULT=$(xcrun altool --notarization-info "$REQUEST_UUID" \
+ --username "$APPLE_ID_USER" \
+ --password @env:APPLE_ID_PASSWORD \
+ --output-format xml)
+ STATUS=$(echo "$RESULT" | xpath \
+ "//key[normalize-space(text()) = 'Status']/following-sibling::string[1]/text()" 2> /dev/null)
+
+ if [ "$STATUS" = "success" ]; then
+ echo "Notarization of $APP_BUNDLE succeeded!"
+ break
+ elif [ "$STATUS" = "in progress" ]; then
+ echo "Notarization in progress..."
+ sleep 20
+ else
+ echo "Notarization of $APP_BUNDLE failed:"
+ echo "$RESULT"
+ exit 1
+ fi
+done
+
+# Staple the notary ticket
+xcrun stapler staple "$APP_BUNDLE"