[PATCH RFC] Build system changes for macOS

[Darell Tan]

Hi,

I'm trying to get busybox to compile on macOS and I have encountered a
few problems. I have attached a patch that is my take on a solution.
Feedback and suggestions are welcome. I hope to get this patch merged
to avoid maintaining it out-of-tree.

One of the problems is that the archiver "ar" shipped by Apple doesn't
like creating empty archives, so something like "ar rcs foo.a" will
fail. I tried working around that by just "touch foo.a" but apparently
during linking, it doesn't work with empty files as well.

The problem is best solved by Kbuild not creating empty built-in.o and
lib.a files, and also to not pull in those files during linking, but
I'm not sure how to solve this completely or cleanly. It seems that
the builtin-target seems to depend on $(lib-target) being non-empty,
which is always the case, so I removed that. The other half of the
solution is having "scripts/trylink" drop empty or non-existent
$A_FILES during linking.

Aside from that, Apple's compiler/linker also doesn't support a few
gcc flags, which are checked for and excluded by this patch, like
-static-libgcc, --start-group/--end-group, --warn-common, and
--verbose.

The last oddity is that the compiler doesn't find symbols inside
archives. This problem only shows up during the linking stage, when
object files reference ash_ptr_to_globals_misc for example, which is
in the same archive file. I followed the solution in a StackOverflow
post [1], which is to run "ranlib -c" on the archive files.

With these changes, busybox can be successfully built after fixing up
platform differences like HAVE_MNTENT, and some other changes that can
be made in .config like MONOTONIC_SYSCALL. I'll leave those for a
separate patch.

I have tested this patch on Ubuntu 18 and macOS 10.13.

Thanks.

[1] https://stackoverflow.com/questions/34595766/object-files-not-properly-added-to-archive-on-mac

Regards,
Darell Tan

----------
 Makefile.flags         |  5 ++++-
 scripts/Makefile.build |  4 ++--
 scripts/trylink        | 23 +++++++++++++++++++++++
 3 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/Makefile.flags b/Makefile.flags
index 6f6142cc5..e9c994eaa 100644
--- a/Makefile.flags
+++ b/Makefile.flags
@@ -51,7 +51,10 @@ CFLAGS += $(call cc-option,-fno-builtin-strlen
-finline-limit=0 -fomit-frame-poi
 # -fno-guess-branch-probability: prohibit pseudo-random guessing
 # of branch probabilities (hopefully makes bloatcheck more stable):
 CFLAGS += $(call cc-option,-fno-guess-branch-probability,)
-CFLAGS += $(call cc-option,-funsigned-char -static-libgcc,)
+CFLAGS += $(call cc-option,-funsigned-char,)
+ifneq ($(shell echo '__APPLE__' | $(CC) -E -xc - | tail -n 1),1)
+CFLAGS += $(call cc-option,-static-libgcc,)
+endif
 CFLAGS += $(call cc-option,-falign-functions=1 -falign-jumps=1
-falign-labels=1 -falign-loops=1,)
 # Defeat .eh_frame bloat (gcc 4.6.3 x86-32 defconfig: 20% smaller
busybox binary):
 CFLAGS += $(call cc-option,-fno-unwind-tables,)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 5eac45f91..846ed8fce 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -79,7 +79,7 @@ ifneq ($(strip $(lib-y) $(lib-m) $(lib-n) $(lib-)),)
 lib-target := $(obj)/lib.a
 endif

-ifneq ($(strip $(obj-y) $(obj-m) $(obj-n) $(obj-) $(lib-target)),)
+ifneq ($(strip $(obj-y) $(obj-m) $(obj-n) $(obj-)),)
 builtin-target := $(obj)/built-in.o
 endif

@@ -271,7 +271,7 @@ endif # builtin-target
 #
 ifdef lib-target
 quiet_cmd_link_l_target = AR      $@
-cmd_link_l_target = rm -f $@; $(AR) $(EXTRA_ARFLAGS) rcs $@ $(lib-y)
+cmd_link_l_target = rm -f $@; [ -n "$(lib-y)" ] && $(AR)
$(EXTRA_ARFLAGS) rcs $@ $(lib-y)

 $(lib-target): $(lib-y) FORCE
  $(call if_changed,link_l_target)
diff --git a/scripts/trylink b/scripts/trylink
index bb6b2de2f..5acf60c96 100755
--- a/scripts/trylink
+++ b/scripts/trylink
@@ -89,12 +89,35 @@ if ! check_cc "-Wl,--sort-section,alignment"; then
     SORT_SECTION=""
 fi

+$CC --version 2>/dev/null | grep -q Apple
+IS_APPLE=$?
+
 START_GROUP="-Wl,--start-group"
 END_GROUP="-Wl,--end-group"
 INFO_OPTS() {
+ [ $IS_APPLE -eq 0 ] && \
+ echo "-Wl,-map,$EXE.map" || \
  echo "-Wl,--warn-common -Wl,-Map,$EXE.map -Wl,--verbose"
 }

+if ! check_cc "$START_GROUP $END_GROUP"; then
+    echo "Your linker does not support --start-group / --end-group"
+    START_GROUP=
+    END_GROUP=
+fi
+
+# remove non-existent A_FILES here
+# built-in.o files get passed here too, actually
+NEW_A_FILES=
+for f in $A_FILES; do
+ [ -s "$f" ] || continue
+ NEW_A_FILES="$NEW_A_FILES $f"
+
+ # get ranlib to "index" archive on macOS
+ [ $IS_APPLE -eq 0 ] && ranlib -c "$f" 2>/dev/null
+done
+A_FILES="$NEW_A_FILES"
+
 # gold may not support --sort-common (yet)
 SORT_COMMON="-Wl,--sort-common"
 if ! check_cc "-Wl,--sort-common"; then