[HN Gopher] Achieving an open-source implementation of Apple Cod...
___________________________________________________________________
Achieving an open-source implementation of Apple Code Signing and
notarization
Author : indygreg2
Score : 201 points
Date : 2022-08-08 15:34 UTC (7 hours ago)
(HTM) web link (gregoryszorc.com)
(TXT) w3m dump (gregoryszorc.com)
| deknos wrote:
| is there a public format for Code or Keysigning Servers? Like
| SCEP/EST, but for other stuff?
| infotogivenm wrote:
| I've always used gon ( https://github.com/mitchellh/gon ) for
| this, which is open source golang, but I don't think it supports
| mach-o embedding. I'll have to try this tool out.
| nimbius wrote:
| this feels like a job for lemur and smallstep
|
| https://smallstep.com/docs/step-ca
|
| https://github.com/Netflix/lemur
| ridiculous_fish wrote:
| I think Apple's official code signing tool is also open source?
| https://github.com/Apple-FOSS-Mirror/security_systemkeychain...
|
| xar is another tool for codesigning Mac installer packages, which
| runs on Linux. I've used it successfully in the past.
|
| https://users.wfu.edu/cottrell/productsign/productsign_linux...
| bri3d wrote:
| Good pointers indeed -
|
| Apple's code signing tool is open source, but heavily dependent
| on Cocoa/OSX libraries like CoreFoundation. I'm not sure if it
| works with any of the open-source reimplementations of these
| libraries, although I kind of want to try now, just for fun -
| has anyone done this already? Plus, it's not Written in Rust
| (TM)!
|
| `xar` only signs packages, not Mach-O binaries, at least as far
| as I know?
| TazeTSchnitzel wrote:
| CoreFoundation is (partially?) open-source and cross-platform
| now: https://github.com/apple/swift-corelibs-foundation
| bouke wrote:
| That's not CoreFoundation, a C library, but a Swift
| reimplementation of that library, only to be consumed by
| Swift (import Foundation).
| stormbrew wrote:
| CF was always (approximately) a multiplatform and open
| source subset of foundation (which is the thing this
| isn't). I think this is a superset and extension of the
| original core foundation under more favorable license
| terms though.
|
| The "original" CF can be found here in various revisions
| tied to macos versions, complete with theoretically
| useful Linux makefiles and apple public source license
| files: https://opensource.apple.com/source/CF/
|
| (Credentials: I worked on the swift Linux port and made
| an effort to use these very libraries before the swift
| project itself took on the task of porting CF to swift.
| While I was working on it we mostly just ifdef'd out
| everything to do with foundation when targeting Linux)
| indygreg2 wrote:
| XAR signing is effectively just an RFC 5652 CMS signature
| plus some minimal data structure manipulation. Code at https:
| //github.com/indygreg/PyOxidizer/blob/faa7dfcea5d66bf5....
|
| Mach-O and bundles, by contrast, require a myriad of
| additional data structures requiring thousands of lines of
| code to support. To my knowledge, nobody else has implemented
| signing of these far-more-complicated primitives. (Existing
| Mach-O signing solutions just do ad-hoc signing and/or don't
| handle Mach-O in the context of a bundle.)
| bogwog wrote:
| > Plus, it's not Written in Rust (TM)!
|
| It could be co-authored by Xi Jinping and Mark Zuckerberg in
| COBOL and hosted on NPM, and that wouldn't stop me from using
| an open source codesign alternative that runs on Linux.
| thewebcount wrote:
| Can you elaborate? Since I mostly work on macOS stuff, I
| don't understand the workflow that leads to needing this
| sort of tool. What's the process of writing software for
| either macOS or iOS that doesn't involve needing macOS or
| iOS at some point? Like you probably need to test on those
| systems if you're supporting software for them, right? Is
| it because development is done on Linux, and only testing
| is done on, say, macOS? Sorry if this is a stupid question.
| I just haven't run into this sort of situation before and
| am trying to better understand it.
| adastra22 wrote:
| Bitcoin Core signs it's own binaries in a deterministic
| environment. Not sure if they use xar, or what.
| giomasce wrote:
| What does notarization mean here?
| salawat wrote:
| Realistically? It's an additional static analysis hoop to jump
| through and another brick in Apple's walled garden in terms of
| trying to lock-out generic programs written by anyone who can't
| be arsed to fiddle with cryptography.
| duped wrote:
| https://developer.apple.com/documentation/security/notarizin...
| candiddevmike wrote:
| Will this upload to the app store?
| Sytten wrote:
| The only thing left to solve is to actually build cross platform.
| You can do with rust cross but you have to build your own docker
| images on macos first since apple licensing prohibits
| redistribution of binaries. It would save us so much money in
| github CI minutes.
| roblabla wrote:
| This is actually a solved problem, using osxcross[0]. The
| experience is honestly very smooth, and we don't require _any_
| apple proprietary binaries. The only thing apple-proprietary is
| their SDK (containing the header files for compiling, and tbd
| files for linking), which can be downloaded from apple 's
| website (at least if you have a developer account), or from
| various GitHub projects archiving them.
|
| [0]: https://github.com/tpoechtrager/osxcross
| funhatch wrote:
| https://pmbaty.com/iosbuildenv/
| bogwog wrote:
| You mean building mac/ios apps on Linux?
|
| That's relatively easy to do, with the only
| questionable/legally gray area being access to the iOS/Mac
| SDKs. I'm not a lawyer, but it's my understanding that the
| Google v Oracle case established that APIs aren't protected by
| copyright. So if we can use the headers from e.g. the iOS SDK
| on Linux wtohout worrying about copyright issues (Apple
| developer agreements may be an issue though), it should be
| possible to autogenerate stub libraries for linking purposes.
|
| With that, you can compile and link a binary for iOS or Mac on
| Linux.
|
| Next, you'll need to implement the packaging and signing
| portion. Packaging (generating an app bundle) is easy, but
| codesigning requires Apple's codesign utility, which doesn't
| work outside of Mac, even with a project like Darling (last I
| checked, anyways).
|
| But if the OP does implement a working codesign alternative,
| then that means it should be possible to create an iOS/Mac app
| that can be distributed on the App Store entirely from Linux
| with no Xcode, virtual machines, cloud macs, etc.
|
| The only gate left then is the Apple tax/signing certificate
| you need to buy for $99/year. But that might change with all of
| the antitrust pressure Apple is facing, including the recent EU
| law that would force Apple to allow competing app stores.
| TazeTSchnitzel wrote:
| Zig already does this I think, they committed the Apple
| headers to their repo.
| tssva wrote:
| "I'm not a lawyer, but it's my understanding that the Google
| v Oracle case established that APIs aren't protected by
| copyright."
|
| I'm no lawyer either but I have read the Supreme Court's
| decision in the case. It doesn't address whether APIs are
| protected by copyright. The Supreme Court generally makes the
| most limited ruling needed in order to resolve a case. In
| this case there were two issues at play.
|
| -Are API's copyrightable? -If so, was Google's use of the
| APIs fair use under copyright law as a jury had previously
| ruled?
|
| In order to make the most limited ruling needed to decide the
| case the Supreme Court first addressed the fair use question
| with an assumption that APIs could be copyrighted. The Court
| decided that Google's use did fall under fair use and so the
| case could be decided by the fair use question alone. They
| therefore didn't bother to address the question of whether
| APIs can be copyrighted.
|
| As it stands right now there is a U.S. Court of Appeals
| ruling in the Oracle case that APIs are copyrightable which
| until the Supreme Court does address the issue means they are
| as far as the courts are concerned. Fair use is a case-by-
| case defense and the Supreme Court's decision doesn't mean
| that all use of an API is fair use. This is all very much
| still all up in the air and any use of the headers from the
| iOS SDK should be concerned about copyright issues.
| neilk wrote:
| Kudos to these developers. I wish them success.
|
| If I can do a small derail: I and some contributors had this
| mostly working a few iOS versions ago with isign
| (https://github.com/isignpy/isign). This is befor notarization.
| Announced several times to HN but didn't seem to be interesting
| to many people.
|
| This was a spin-off from our work at a testing company. I'm not
| an iOS developer. (That may be why I could never see what to do
| with this.)
|
| I tried to make it into something, but I could never figure out
| who actually wanted this. What potential uses are foreseen for
| rcodedesign?
|
| - Weird custom signing. We did a project for a large financial
| company to make it compatible with hardware security module
| signing, but they never implemented it due to the pandemic
| changing priorities, and then someone discovered there was a
| little-known MacOS API to do something similar anyway, so our
| project was shelved. (We got paid though).
|
| - "Alternative" app distribution. From time to time I am
| contacted by people who run alternative app stores, either
| distributing hacked versions of paid apps, or who run app store
| in countries under embargo from Western countries. (They have
| banks, their customers have iPhones, they want a way to
| distribute iOS apps). The ethical issues and legal risks seemed
| significant so I never pursued that. If someone wants to go do
| that, I guess that's a use case.
|
| - Build pipeline. Maybe, but you can't fully build a iOS app on
| Linux so it didn't seem like a win.
|
| - Mass app production. I am aware of some app white-labeling
| concepts that used isign back in the day. There was a guy who had
| a generic app for local news outlets, and essentially uploaded it
| to the App Store dozens of times under different names, for new
| versions. isign was more scriptable (and I wrote a multi-resign
| option for this use case) but I didn't see a way to turn this
| into a business.
|
| - Testing on real devices. Even the original use case (re-signing
| apps to run on local device labs) has been mostly obviated.
| Hosting and scripting on MacOS has gotten better.
|
| - ??? something else? I even reached out to saurik about this at
| a conference. Cydia worked on jailbroken devices, so it didn't
| need signing. To my surprise saurik was angry with me. He said
| that many, many companies had figured this out internally so he
| was skeptical we would ever release it. (We did!)
|
| So... again kudos, but what can we do with this? I'm not dissing
| it, I'm genuinely baffled. I also thought this would be important
| and useful but couldn't figure it out.
| nicoburns wrote:
| Regarding use case:
|
| It could certainly be useful for macOS CLI tools. Languages
| like Go and Zig can cross-compile seamlessly from linux (and
| Rust is nearly but not quite there). With this you could
| compile, sign and notarize a ready-to-publish binary without
| ever touching a macOS machine.
| bogwog wrote:
| I'm kind of confused by what you mean by "re-sign". The repo
| you linked to says that it's a tool to "re-sign" apps. Does
| that mean it requires an already-signed app to function?
|
| Mac/iOS app bundles need to be signed, but so does the actual
| compiled binary within the bundle. The signing process, among
| other things, embeds entitlement information into the binary
| (permissions for using certain APIs). Can isignpy do this? Or
| does it only do app bundle signing?
|
| From the perspective of app development, a tool that can
| perform full code signing on Linux is huge. Imagine how much of
| a pain (and expense) it is to maintain a CI pipeline when you
| need a physical Mac (since you can't legally run MacOS in a
| VM). There are more than a few cloud companies that offer
| dedicated Macs pretty much just for this purpose.
| roblabla wrote:
| > - Build pipeline. Maybe, but you can't fully build a iOS app
| on Linux so it didn't seem like a win.
|
| This is my use case (I implemented notarization API support in
| rcodesign).
|
| I build MacOS apps entirely from Linux using my own bespoke
| toolchain, based on osxcross. It can potentially be used to
| build iOS apps too (I tested it in jest some time ago, but I
| don't use it for anything serious). This is all done without
| any of the apple binaries. The toolchain uses nix to pull
| everything together, clang and other llvm utilities as a
| compiler, lld as a linker, rcodesign for signing and
| notarization, and the macOS/iOS SDK for headers and tbd files
| used for linking. And a bit of custom glue code in bash to hold
| it all together.
|
| The reason I do this is simple: My CI is entirely Linux-based.
| I already cross-build windows binary from Linux. This makes CI
| so much easier, as everything can be containerized, and thus
| run on most standard CI offerings. I only need to bring Windows
| and macOS VMs to run test suites. Because VM bringup is slow
| and somewhat fragile, pushing everything to run in Linux
| containers is a huge win.
|
| It also gives me a lot more control when things inevitably
| break. Being open source, rcodesign can be trivially changed,
| be it to add more logging, or add new features. For instance,
| rcodesign has a remote signing mechanism! This isn't something
| that exists in the normal codesign implementation.
| [deleted]
| xrd wrote:
| This is so fantastic. I wouldn't have believed it was possible
| had I not seen it here. I maintain multiple old OSX machines
| solely for this purpose; migrating to Linux for this step makes
| me feel so much safer for my CI build systems.
| gilgoomesh wrote:
| > There are probably thousands of companies and individuals who
| have wanted to release Apple software from non-macOS operating
| systems. (The existence and popularity of tools like fastlane
| seems to confirm this.)
|
| I'm not sure the existence/popularity of fastlane supports the
| non-macOS operating systems argument since its primary purpose is
| as a wrapper around Apple's clunky/undocumented build and upload
| tools on macOS.
| xet7 wrote:
| How to build GUI apps for iOS and macOS with Rust ?
|
| Can this notarize apps made with other programming languages?
| TazeTSchnitzel wrote:
| You could use Qt and cross-compile.
| indygreg2 wrote:
| rcodesign can sign and notarize applications written in any
| language. From rcodesign's perspective, Rust is an
| implementation detail of the tool itself.
| norman784 wrote:
| As far I can tell, you can compile the binary and sign your
| app.
|
| # From a Git checkout
|
| $ cargo run --bin rcodesign -- --help
|
| $ cargo install --bin rcodesign
|
| # Remote install.
|
| $ cargo install --git https://github.com/indygreg/PyOxidizer
| --branch main --bin rcodesign apple-codesign
| rvz wrote:
| So that is a complicated way of saying 'No.'
|
| The answer still stands to be:
|
| You still cannot build a production ready cross-platform GUI
| app for both iOS and macOS written in Rust which is what the
| parent comment is looking for.
| steveklabnik wrote:
| Your parent is clearly answering the OP's second question
| but not the first, come on.
|
| I'm aware of production iOS apps using Rust that are
| either:
|
| 1. Using rust for core logic but still using Swift/obj c
| for UI.
|
| 2. Pure Rust but don't use native widgets. Like games or
| similar custom UI.
|
| That said it is in my understanding theoretically possible
| but I'm not aware of a paved cow path here. It's also not
| my area of specialty though.
___________________________________________________________________
(page generated 2022-08-08 23:00 UTC)