tmac build: conform to macOS 10.15 Gatekeeper requirements - electrum - Electrum Bitcoin wallet (HTM) git clone https://git.parazyd.org/electrum (DIR) Log (DIR) Files (DIR) Refs (DIR) Submodules --- (DIR) commit 095464b620e3e1bb699d174d969526f78ad356b9 (DIR) parent 9baaf1afda3181cd511fda83c0be426a85c02abd (HTM) Author: SomberNight <somber.night@protonmail.com> Date: Tue, 12 May 2020 12:37:40 +0200 mac build: conform to macOS 10.15 Gatekeeper requirements fixes #6128 some of this is based on: https://github.com/metabrainz/picard/blob/e1354632d2db305b7a7624282701d34d73afa225/scripts/package/macos-notarize-app.sh https://github.com/Electron-Cash/Electron-Cash/commit/1eb8b71e7d11141432f1c46629683a5a703795e2 https://github.com/Electron-Cash/Electron-Cash/commit/24e44e9784fa23fa9f408ce3f9489fac8568093b https://github.com/Electron-Cash/Electron-Cash/commit/5abec73eee0cdeb725e3c5a989621ec4ccfb92a0 Diffstat: M contrib/osx/README.md | 65 ++++++++++++++++--------------- D contrib/osx/base.sh | 23 ----------------------- A contrib/osx/entitlements.plist | 19 +++++++++++++++++++ M contrib/osx/make_osx | 62 +++++++++++++++++++++++-------- A contrib/osx/notarize_app.sh | 77 +++++++++++++++++++++++++++++++ 5 files changed, 175 insertions(+), 71 deletions(-) --- (DIR) diff --git a/contrib/osx/README.md b/contrib/osx/README.md t@@ -1,5 +1,5 @@ -Building Mac OS binaries -======================== +Building macOS binaries +======================= ✗ _This script does not produce reproducible output (yet!). Please help us remedy this._ t@@ -7,36 +7,47 @@ Building Mac OS binaries This guide explains how to build Electrum binaries for macOS systems. -## 1. Building the binary +## Building the binary -This needs to be done on a system running macOS or OS X. We use El Capitan (10.11.6) as building it -on High Sierra (or later) -makes the binaries [incompatible with older versions](https://github.com/pyinstaller/pyinstaller/issues/1191). +This needs to be done on a system running macOS or OS X. -Another factor for the minimum supported macOS version is the -[bundled Qt version](https://github.com/spesmilo/electrum/issues/3685). +Notes about compatibility with different macOS versions: +- In general the binary is not guaranteed to run on an older version of macOS + than what the build machine has. This is due to bundling the compiled Python into + the [PyInstaller binary](https://github.com/pyinstaller/pyinstaller/issues/1191). +- The [bundled version of Qt](https://github.com/spesmilo/electrum/issues/3685) also + imposes a minimum supported macOS version. +- If you want to build binaries that conform to the macOS "Gatekeeper", so as to + minimise the warnings users get, the binaries need to be codesigned with a + certificate issued by Apple, and starting with macOS 10.15 the binaries also + need to be notarized by Apple's central server. The catch is that to be able to build + binaries that Apple will notarise (due to the requirements on the binaries themselves, + e.g. hardened runtime) the build machine needs at least macOS 10.14. + See [#6128](https://github.com/spesmilo/electrum/issues/6128). + +We currently build the release binaries on macOS 10.14.6, and these seem to run on +10.13 or newer. Before starting, make sure that the Xcode command line tools are installed (e.g. you have `git`). -#### 1.1a Get Xcode +#### 1.a Get Xcode Building the QR scanner (CalinsQRReader) requires full Xcode (not just command line tools). -The last Xcode version compatible with El Capitan is Xcode 8.2.1 - Get it from [here](https://developer.apple.com/download/more/). - Unfortunately, you need an "Apple ID" account. +(note: the last Xcode that runs on macOS 10.14.6 is Xcode 11.3.1) + After downloading, uncompress it. Make sure it is the "selected" xcode (e.g.): sudo xcode-select -s $HOME/Downloads/Xcode.app/Contents/Developer/ -#### 1.1b Build QR scanner separately on newer Mac +#### 1.b Build QR scanner separately on another Mac -Alternatively, you can try building just the QR scanner on newer macOS. +Alternatively, you can try building just the QR scanner on another Mac. On newer Mac, run: t@@ -46,27 +57,17 @@ On newer Mac, run: Move `prebuilt_qr` to El Capitan: `contrib/osx/CalinsQRReader/prebuilt_qr`. -#### 1.2 Build Electrum +#### 2. Build Electrum cd electrum ./contrib/osx/make_osx - -This creates both a folder named Electrum.app and the .dmg file. - -## 2. Building the image deterministically (WIP) -The usual way to distribute macOS applications is to use image files containing the -application. Although these images can be created on a Mac with the built-in `hdiutil`, -they are not deterministic. - -Instead, we use the toolchain that Bitcoin uses: genisoimage and libdmg-hfsplus. -These tools do not work on macOS, so you need a separate Linux machine (or VM). - -Copy the Electrum.app directory over and install the dependencies, e.g.: +This creates both a folder named Electrum.app and the .dmg file. - apt install libcap-dev cmake make gcc faketime - -Then you can just invoke `package.sh` with the path to the app: +If you want the binaries codesigned for MacOS and notarised by Apple's central server, +provide these env vars to the `make_osx` script: - cd electrum - ./contrib/osx/package.sh ~/Electrum.app/ + CODESIGN_CERT="Developer ID Application: Electrum Technologies GmbH (L6P37P7P56)" \ + APPLE_ID_USER="me@email.com" \ + APPLE_ID_PASSWORD="1234" \ + ./contrib/osx/make_osx (DIR) diff --git a/contrib/osx/base.sh b/contrib/osx/base.sh t@@ -1,23 +0,0 @@ -#!/usr/bin/env bash - -. $(dirname "$0")/../build_tools_util.sh - - -function DoCodeSignMaybe { # ARGS: infoName fileOrDirName codesignIdentity - infoName="$1" - file="$2" - identity="$3" - deep="" - if [ -z "$identity" ]; then - # we are ok with them not passing anything; master script calls us unconditionally even if no identity is specified - return - fi - if [ -d "$file" ]; then - deep="--deep" - fi - if [ -z "$infoName" ] || [ -z "$file" ] || [ -z "$identity" ] || [ ! -e "$file" ]; then - fail "Argument error to internal function DoCodeSignMaybe()" - fi - info "Code signing ${infoName}..." - codesign -f -v $deep -s "$identity" "$file" || fail "Could not code sign ${infoName}" -} (DIR) diff --git a/contrib/osx/entitlements.plist b/contrib/osx/entitlements.plist t@@ -0,0 +1,19 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> +<plist version="1.0"> +<dict> + <!-- These are required for binaries built by PyInstaller --> + <!-- see pyinstaller/pyinstaller#4629 --> + <key>com.apple.security.cs.allow-unsigned-executable-memory</key> + <true/> + <key>com.apple.security.cs.disable-library-validation</key> + <true/> + + <!-- These are required for USB HID access (hw wallets). --> + <!-- see https://github.com/Electron-Cash/Electron-Cash/commit/5abec73eee0cdeb725e3c5a989621ec4ccfb92a0 --> + <key>com.apple.security.cs.allow-dyld-environment-variables</key> + <true/> + <key>com.apple.security.cs.allow-jit</key> + <true/> +</dict> +</plist> (DIR) diff --git a/contrib/osx/make_osx b/contrib/osx/make_osx t@@ -5,11 +5,12 @@ PYTHON_VERSION=3.7.6 BUILDDIR=/tmp/electrum-build PACKAGE=Electrum GIT_REPO=https://github.com/spesmilo/electrum -LIBSECP_VERSION="b408c6a8b287003d1ade5709e6f7bc3c7f1d5be7" export GCC_STRIP_BINARIES="1" -. $(dirname "$0")/base.sh + +. $(dirname "$0")/../build_tools_util.sh + CONTRIB_OSX="$(dirname "$(realpath "$0")")" CONTRIB="$CONTRIB_OSX/.." t@@ -24,26 +25,46 @@ which brew > /dev/null 2>&1 || fail "Please install brew from https://brew.sh/ t which xcodebuild > /dev/null 2>&1 || fail "Please install Xcode and xcode command line tools to continue" # Code Signing: See https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html -APP_SIGN="" -if [ -n "$1" ]; then +if [ -n "$CODESIGN_CERT" ]; then # Test the identity is valid for signing by doing this hack. There is no other way to do this. cp -f /bin/ls ./CODESIGN_TEST - codesign -s "$1" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1 + codesign -s "$CODESIGN_CERT" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1 res=$? rm -f ./CODESIGN_TEST if ((res)); then - fail "Code signing identity \"$1\" appears to be invalid." + fail "Code signing identity \"$CODESIGN_CERT\" appears to be invalid." fi unset res - APP_SIGN="$1" - info "Code signing enabled using identity \"$APP_SIGN\"" + info "Code signing enabled using identity \"$CODESIGN_CERT\"" else - warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system as the first argument to this script to enable signing." + warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system to enable signing." fi + +function DoCodeSignMaybe { # ARGS: infoName fileOrDirName + infoName="$1" + file="$2" + deep="" + if [ -z "$CODESIGN_CERT" ]; then + # no cert -> we won't codesign + return + fi + if [ -d "$file" ]; then + deep="--deep" + fi + if [ -z "$infoName" ] || [ -z "$file" ] || [ ! -e "$file" ]; then + fail "Argument error to internal function DoCodeSignMaybe()" + fi + hardened_arg="--entitlements=${CONTRIB_OSX}/entitlements.plist -o runtime" + + info "Code signing ${infoName}..." + codesign -f -v $deep -s "$CODESIGN_CERT" $hardened_arg "$file" || fail "Could not code sign ${infoName}" +} + + info "Installing Python $PYTHON_VERSION" export PATH="~/.pyenv/bin:~/.pyenv/shims:~/Library/Python/3.7/bin:$PATH" -if [ -d "~/.pyenv" ]; then +if [ -d "${HOME}/.pyenv" ]; then pyenv update else curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash > /dev/null 2>&1 t@@ -109,7 +130,7 @@ rm -fr build # prefer building using xcode ourselves. otherwise fallback to prebuilt binary xcodebuild || cp -r prebuilt_qr build || fail "Could not build CalinsQRReader" popd -DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop +DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app" info "Installing requirements..." t@@ -131,7 +152,7 @@ for d in ~/Library/Python/ ~/.pyenv .; do done info "Building binary" -APP_SIGN="$APP_SIGN" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary" +APP_SIGN="$CODESIGN_CERT" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary" info "Adding bitcoin URI types to Info.plist" plutil -insert 'CFBundleURLTypes' \ t@@ -139,14 +160,23 @@ plutil -insert 'CFBundleURLTypes' \ -- dist/$PACKAGE.app/Contents/Info.plist \ || fail "Could not add keys to Info.plist. Make sure the program 'plutil' exists and is installed." -DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop +DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app" + +if [ ! -z "$CODESIGN_CERT" ]; then + if [ ! -z "$APPLE_ID_USER" ]; then + info "Notarizing .app with Apple's central server..." + "${CONTRIB_OSX}/notarize_app.sh" "dist/${PACKAGE}.app" || fail "Could not notarize binary." + else + warn "AppleID details not set! Skipping Apple notarization." + fi +fi info "Creating .DMG" hdiutil create -fs HFS+ -volname $PACKAGE -srcfolder dist/$PACKAGE.app dist/electrum-$VERSION.dmg || fail "Could not create .DMG" -DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg" "$APP_SIGN" # If APP_SIGN is empty will be a noop +DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg" -if [ -z "$APP_SIGN" ]; then +if [ -z "$CODESIGN_CERT" ]; then warn "App was built successfully but was not code signed. Users may get security warnings from macOS." - warn "Specify a valid code signing identity as the first argument to this script to enable code signing." + warn "Specify a valid code signing identity to enable code signing." fi (DIR) diff --git a/contrib/osx/notarize_app.sh b/contrib/osx/notarize_app.sh t@@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# from https://github.com/metabrainz/picard/blob/e1354632d2db305b7a7624282701d34d73afa225/scripts/package/macos-notarize-app.sh + + +if [ -z "$1" ]; then + echo "Specify app bundle as first parameter" + exit 1 +fi + +if [ -z "$APPLE_ID_USER" ] || [ -z "$APPLE_ID_PASSWORD" ]; then + echo "You need to set your Apple ID credentials with \$APPLE_ID_USER and \$APPLE_ID_PASSWORD." + exit 1 +fi + +APP_BUNDLE=$(basename "$1") +APP_BUNDLE_DIR=$(dirname "$1") + +cd "$APP_BUNDLE_DIR" || exit 1 + +# Package app for submission +echo "Generating ZIP archive ${APP_BUNDLE}.zip..." +ditto -c -k --rsrc --keepParent "$APP_BUNDLE" "${APP_BUNDLE}.zip" + +# Submit for notarization +echo "Submitting $APP_BUNDLE for notarization..." +RESULT=$(xcrun altool --notarize-app --type osx \ + --file "${APP_BUNDLE}.zip" \ + --primary-bundle-id org.electrum.electrum \ + --username $APPLE_ID_USER \ + --password @env:APPLE_ID_PASSWORD \ + --output-format xml) + +if [ $? -ne 0 ]; then + echo "Submitting $APP_BUNDLE failed:" + echo "$RESULT" + exit 1 +fi + +REQUEST_UUID=$(echo "$RESULT" | xpath \ + "//key[normalize-space(text()) = 'RequestUUID']/following-sibling::string[1]/text()" 2> /dev/null) + +if [ -z "$REQUEST_UUID" ]; then + echo "Submitting $APP_BUNDLE failed:" + echo "$RESULT" + exit 1 +fi + +echo "$(echo "$RESULT" | xpath \ + "//key[normalize-space(text()) = 'success-message']/following-sibling::string[1]/text()" 2> /dev/null)" + +# Poll for notarization status +echo "Submitted notarization request $REQUEST_UUID, waiting for response..." +sleep 60 +while : +do + RESULT=$(xcrun altool --notarization-info "$REQUEST_UUID" \ + --username "$APPLE_ID_USER" \ + --password @env:APPLE_ID_PASSWORD \ + --output-format xml) + STATUS=$(echo "$RESULT" | xpath \ + "//key[normalize-space(text()) = 'Status']/following-sibling::string[1]/text()" 2> /dev/null) + + if [ "$STATUS" = "success" ]; then + echo "Notarization of $APP_BUNDLE succeeded!" + break + elif [ "$STATUS" = "in progress" ]; then + echo "Notarization in progress..." + sleep 20 + else + echo "Notarization of $APP_BUNDLE failed:" + echo "$RESULT" + exit 1 + fi +done + +# Staple the notary ticket +xcrun stapler staple "$APP_BUNDLE"