Worked around bug/quirk in some EFIs that prevented filesystem drivers

[refind] / BUILDING.txt

diff --git a/BUILDING.txt b/BUILDING.txt
index f9fe9a29eb9e5e048196d7ff58097f7e019ccc39..14096c5a81e7348180169eadf89b755ef13a73f0 100644 (file)
--- a/BUILDING.txt
+++ b/BUILDING.txt
@@ -1,57 +1,168 @@
-From rEFIt to rEFInd
-====================
-
-rEFInd is derived from rEFIt (http://refit.sourceforge.net), but the two
-programs support different build environments. rEFIt was created with
-Intel's EFI Application Toolkit
-(http://www.intel.com/technology/efi/toolkit_overview.htm) or TianoCore's
-EFI Toolkit (https://efi-toolkit.tianocore.org), along with Microsoft's
-Visual C compiler.
-
-Compiling the source code provided on the rEFIt site under Linux never
-worked for me, although the documentation claimed it would. Apparently
-other Linux developers have run into the same problem; Debian provides a
-rEFIt package (http://packages.debian.org/sid/refit) that includes
-extensive patches to enable the program to compile under Linux using the
-GNU-EFI package (http://sourceforge.net/projects/gnu-efi/). Although
-GNU-EFI is less sophisticated than recent versions of TianoCore's toolkit,
-GNU-EFI is my preferred environment because I prefer working in Linux and
-because TianoCore's toolkit relies on obsolete versions of Microsoft's
-Visual C++, and I had trouble tracking that down. For this reason, I used
-Debian's patched version of rEFIt as a starting point in forking rEFInd.
-The main down side to doing this is that a few rEFIt features got lost.
-Most notable among these is the ability to load filesystem drivers. I've
-also dropped ancillary programs, such as the ext2fs driver module and
-gptsync program, from rEFInd. You can still use these tools with rEFInd,
-but you'll need to install them separately; and in the case of filesystem
-drivers, you'll need to configure your EFI to load them before you launch
-rEFInd.
-
-

 Requirements
 ============
 
 To compile rEFInd, you'll need the following:
 
 * A Linux installation. Note that this installation does NOT need to be
 Requirements
 ============
 
 To compile rEFInd, you'll need the following:
 
 * A Linux installation. Note that this installation does NOT need to be

-  EFI-based. It can be 32- or 64-bit, but unless you use a cross-compiler
-  (which I've not tested), it must be the appropriate bit width for your
-  EFI implementation. (Normally that means 64-bit.) If you don't normally
-  run Linux, you can run it in a VirtualBox or similar virtual machine.
+  EFI-based. It can use IA32 (aka x86, i386, or other things), X64 (aka
+  x86-64, AMD64, or EM64T), or AA64 (aka AARCH64 or ARM64), but unless you
+  use a cross-compiler, it must use the same CPU type and bit depth as your
+  EFI implementation. (Normally that means 64-bit X64.) If you don't
+  normally run Linux, you can run it in a VirtualBox or similar virtual
+  machine. (I describe some unsupported non-Linux build options shortly.)
+
+* A standard set of Linux development tools, based on GCC. (I've tried
+  using clang 3.4 under Ubuntu, with partial success. The main rEFInd
+  binary, gptsync, and some drivers compile successfully; but only gptsync
+  runs normally. The drivers I've tried and the main rEFInd binary crash.)
+
+* One of the following:
+
+  * The TianoCore EDK2 package
+    (http://sourceforge.net/projects/tianocore/). I initially used the
+    UDK2010 package and others in that series, but beginning with rEFInd
+    0.8.2, I've been using UDK2014
+    (http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=UDK2014).
+    All of the UDK are "frozen," rather than the main EDK2 development
+    branch, which is changing as the developers add features, fix bugs, and
+    so on. See below for TianoCore setup instructions.
+
+  * The GNU-EFI package (http://sourceforge.net/projects/gnu-efi/). You can
+    install this from a package called "gnu-efi"; however, rEFInd relies on
+    features that were added sometime between version 3.0s and 3.0u, so I
+    recommend using 3.0u (or conceivably later). You should check your
+    GNU-EFI version number; you may need to download the latest source
+    code, compile it, and install it locally. The Makefiles assume a
+    GNU-EFI package installed via a package manager. If you install from
+    source code, you may need to adjust those Makefiles' paths.
+
+Of the two toolkits, I prefer to use TianoCore because it produces binaries
+that are about 5-30KiB smaller than those made by GNU-EFI, and I can easily
+build 32-bit binaries on my 64-bit Linux installations. Also, I've had
+problems on a 32-bit Mac Mini with the drivers produced by GNU-EFI hanging
+the system. (I haven't encountered this problem on UEFI-based PCs.) That
+said, the TianoCore EDK2 package is much harder to install, so you may
+prefer to use GNU-EFI unless you have a specific need for the TianoCore
+toolkit. Automated build tools like the OpenSUSE Build Service (OBS) and
+the Ubuntu Personal Package Archive (PPA) mechanism don't yet support
+TianoCore.
+
+It's possible to use a non-Linux platform to compile rEFInd. To the best of
+my knowledge, the rEFInd code doesn't rely on anything Linux-specific in
+its build requirements, and GNU-EFI's Sourceforge page indicates that it
+works under Windows and OS X, too; however, my one attempt to compile
+GNU-EFI under OS X failed. I've received one report that rEFInd compiles
+successfully with Clang and the TianoCore toolkit under OS X by adding the
+refind.inf file to a .dsc file that you use for your own projects. You can
+find brief instructions here (note that this is not my documentation):
+
+https://github.com/snarez/refind-edk2
+
+Under Windows, you would need to either create a project or Makefile for
+your non-GCC compiler or use a GCC port, such as MinGW
+(http://www.mingw.org). You'd probably need to adjust the Makefiles in the
+latter case. A procedure similar to that used under OS X might work using
+GCC or Microsoft's C compiler, but I haven't tested this.
+
+
+Preparing Your Development Kit
+==============================
+
+If you're using Linux, GNU-EFI is the easiest way to compile rEFInd. I
+don't describe GNU-EFI's setup here because it's likely to be fairly easy.
+If your distribution provides a recent enough version, you should be able
+to install a package called gnu-efi and be done with it. If not, you'll
+need to download the source code tarball, build it, and install it. This
+process is fairly typical of Linux packages. Read the GNU-EFI documentation
+if you need help. If you're using GNU-EFI, you can skip the rest of this
+section.
+
+You might want to use the TianoCore toolkit if you have problems with
+GNU-EFI or if you want to build rEFInd on a non-Linux platform.
+Unfortunately, the TianoCore toolkit is weird by Linux programming
+standards. It's also quite large -- it's intended as a means to develop a
+complete EFI firmware implementation, so it contains much more code than is
+needed to develop standalone EFI applications. I don't know of any Linux
+distribution packages for it in RPM, Debian package file, or other formats;
+you MUST install the kit from source code using its own unusual compilation
+procedure. The installation documentation also omits at least one step and
+is a bit unclear about others. Here's how I installed the toolkit:
+
+1) Download UDK2014.SR1.UP1.P1 from
+   https://sourceforge.net/apps/mediawiki/tianocore/index.php?title=UDK2014.
+   Note that UDK2015 is now available, but I have not yet adapted rEFInd to
+   build with it. (UDK2015 has made changes that require matching changes
+   to rEFInd.)
+
+2) Type "mkdir /usr/local/UDK2014". You can use another directory, but the
+   rEFInd Makefile assumes this location. You'll need to edit the EDK2BASE
+   variable in the top-level Makefile if you install somewhere else.
+
+3) Type "cd /usr/local/UDK2014".
+
+4) Unzip the downloaded file (UDK2014.SR1.UP1.P1.Complete.MyWorkSpace.zip)
+   in the current directory (/usr/local/UDK2014). This creates a handful of
+   files, including a tarball and a couple of .zip files.
+
+5) Type "unzip UDK2014.SR1.UP1.MyWorkSpace.zip". This extracts the
+   platform-neutral portion of the development kit.
+
+6) Type "cd MyWorkSpace".
+
+7) Type "tar xvf ../BaseTools\(Unix\).tar". This extracts the
+   Linux/Unix-specific portions of the toolkit.
+
+8) Follow the build instructions at
+   https://sourceforge.net/apps/mediawiki/tianocore/index.php?title=Using_EDK_II_with_Native_GCC_4.4;
+   however, a few changes are required, as detailed below....
+
+9) Type "source edksetup.sh BaseTools". This sets up some environment
+   variables, so subsequent steps (NOT including compiling rEFInd or its
+   drivers) must be typed in the shell you use for this step.

 
 

-* A standard set of Linux development tools, based on GCC.
+10) Edit Conf/target.txt and change the following:
+    - ACTIVE_PLATFORM = MdePkg/MdePkg.dsc
+    - TARGET = RELEASE (DEBUG might work, but I've not tested it).
+    - TARGET_ARCH = X64 (on x86-64; leave this as IA32 on x86 or change it
+      to AARCH64 on ARM64). If you plan to build multiple architectures,
+      you can set this to "IA32 X64" or some other combination.
+    - TOOL_CHAIN_TAG = GCC48 (or other value depending on your GCC version;
+      type "gcc -v" to learn your GCC version number). Note that support
+      for the latest GCC version takes a while to make it into the
+      TianoCore toolkit, so if you're using a very recent GCC, you may need
+      to specify an earlier version and hope for the best or modify
+      Conf/target.txt, as described shortly.
+    The TianoCore Makefiles read some of these variables from this file
+    and use them when accessing directories, so be sure to type these
+    entries in the case specified.

 
 

-* The GNU-EFI package (http://sourceforge.net/projects/gnu-efi/). You
-  can usually install this from a package called "gnu-efi".
+11) The documentation refers to editing Conf/tools_def.txt in addition to
+    Conf/target.txt, but doesn't specify what to change in
+    Conf/tools_def.txt. I haven't found it necessary to make any changes in
+    Conf/tools_def.txt EXCEPT when using GCC 4.7 on a Fedora 17 system with
+    the original UDK2014. With this setup, GCC 4.7 was newer than the most
+    recent GCC that TianoCore supported at that time. With that setup, I
+    found it necessary to change the following line:
+      *_GCC46_X64_ASM_FLAGS            = DEF(GCC46_ASM_FLAGS) -m64 -melf_x86_64
+      to:
+      *_GCC46_X64_ASM_FLAGS            = DEF(GCC46_ASM_FLAGS) -m64
+    Something similar may be necessary if you're using a very recent
+    GCC or some other compiler.

 
 

-It's possible that you could use a non-Linux platform to compile rEFInd. To
-the best of my knowledge, the rEFInd code doesn't rely on anything
-Linux-specific in its build requirements, and GNU-EFI's Sourceforge page
-indicates that it works under Windows and OS X, too. Thus, you may be able
-to compile it on these platforms, but I've not tested it in this way. Under
-Windows, you would need to either create a project or Makefile for your
-non-GCC compiler or use a GCC port, such as MinGW (http://www.mingw.org).
-You'd probably need to adjust the Makefile in the latter case.
+12) Type "make -C /usr/local/UDK2014/MyWorkSpace/BaseTools/Source/C".
+    (This step is not documented on the EDK Web page.) Note that this
+    requires the g++ compiler and UUID development libraries.
+    
+13) Type "build" to build the main set of EDK2 files. This process is
+    likely to take a few minutes. This step requires Python 2; if you have
+    Python 3 installed, you may need to adjust the default python for this
+    build (for instance, by typing "eselect python set python2.7" in
+    Gentoo).
+
+If you installed in a location other than the one I've specified, you must
+edit the EDK2BASE variable in the top-level Makefile in the rEFInd source
+package. Once the toolkit is installed, you can build the filesystem
+drivers or rEFInd, as described below.

 
 
 Compiling rEFInd
 
 
 Compiling rEFInd


@@ -66,33 +177,161 @@ With your development system set up, you can compile rEFInd as follows:
 
 3) Change into the archive's main directory. You should see several files
    including this BUILDING.txt file and several subdirectories such as
 
 3) Change into the archive's main directory. You should see several files
    including this BUILDING.txt file and several subdirectories such as

-   "refind", "libeg", and "include".
+   "refind", "libeg", "mok", "filesystems", and "include".
+
+4) Type "make" to build rEFInd. The Makefile checks for the TianoCore
+   toolkit and tries to use it if it's present. If both toolkits are
+   installed, you can specify the toolkit name -- "make gnuefi" to build
+   with GNU-EFI, or either "make tiano" to build with TianoCore. With any
+   luck, rEFInd will compile without error, leaving the "refind_ia32.efi",
+   "refind_x64.efi", or "refind_aa64.efi" file, depending on your platform,
+   in the "refind" subdirectory. This same step builds the
+   "gptsync_ia32.efi", "gptsync_x64.efi", or "gptsync_aa64.efi" program
+   file, in the "gptsync" subdirectory. (When cross-compiling with
+   TianoCore, "gptsync_aa64.efi" is not built because the cross-compiler
+   failed for me. Since gptsync is likely to be useless on ARM64, this is
+   no great loss.) If you want to build IA32 binaries on an x86-64 (X64)
+   system, type "ARCH=ia32 make". Similarly, you can specify "ARCH=aarch64"
+   to cross-compile for ARM64. This works only if you're using the
+   TianoCore build kit, and only if you set TARGET_ARCH to the appropriate
+   value in target.txt when you set up the TianoCore toolkit. If you plan
+   to build multiple architectures, be sure to copy the .efi file for the
+   first build out of the refind subdirectory before building the second
+   architecture.

 
 

-4) Type "make". With any luck, rEFInd will compile without error, leaving
-   the "refind.efi" file in the "refind" subdirectory.
+5) The default build process does NOT build the filesystem drivers. If you
+   want to build them, you must type "make fs" in the main rEFInd source
+   directory. This command builds with the TianoCore toolkit if it's
+   available and with GNU-EFI if it's not. Alternatively, you can type
+   "make fs_gnuefi" to build with GNU-EFI or "make fs_tiano" to build with
+   TianoCore. (You can prepend "ARCH=ia32" or "ARCH=aarch64" to
+   cross-compile for those architectures, as when building the main rEFInd
+   binary.) The result is filesystem drivers in the filesystems
+   subdirectory, and also copies placed in the drivers_{arch} subdirectory.

 
 If rEFInd doesn't compile correctly, you'll need to track down the source
 of the problem. Double-check that you've got all the necessary development
 
 If rEFInd doesn't compile correctly, you'll need to track down the source
 of the problem. Double-check that you've got all the necessary development

-tools installed, including GCC, make, and GNU-EFI. You may also need to
-adjust the Makefile or Make.common file for your system. For instance, on
-Fedora, you must  change the following variables in Make.common as shown:
+tools installed, including GCC, make, and either GNU-EFI or TianoCore EDK2.
+You may also need to adjust the Makefile or Make.common file; or possibly
+Make* files in code subdirectories. (The main Makefile controls the process
+for both toolkits, while Make.common holds most common options.) The most
+likely thing you'll need to change is the path to the various GNU-EFI
+include files and libraries. Since rEFInd 0.6.2, the default Make.common
+file includes the following definitions:
+
+EFIINC          = /usr/include/efi
+GNUEFILIB       = /usr/lib
+EFILIB          = /usr/lib
+EFICRT0         = /usr/lib
+
+If you've installed GNU-EFI from source code, you may need to add "local"
+to those paths, as in "/usr/local/include/efi". You might need to change
+references to "lib" to "lib32" or "lib64" on some systems. Recall that you
+need at least GNU-EFI version 3.0l to build rEFInd, and until very
+recently, most distributions provided out-of-date versions of this package.
+
+If you're using TianoCore's EDK2, as noted earlier, you may need to adjust
+the EDK2BASE variable in Makefile.
+
+When I tried to compile rEFInd under Ubuntu 12.04 (i386) using GNU-EFI,
+even with a locally-compiled GNU-EFI 3.0p or 3.0q, I got errors like this:

 
 

-GNUEFILIB       = /usr/lib64
-EFILIB          = /usr/lib64
-EFICRT0         = /usr/lib64/gnuefi
+main.o: In function `StartLegacy.isra.0':
+main.c:(.text+0x8b1): undefined reference to `__stack_chk_fail_local'
+lib.o: In function `ScanVolumeBootcode.part.3':
+lib.c:(.text+0xf2f): undefined reference to `__stack_chk_fail_local'
+lib.o: In function `ScanExtendedPartition.isra.4':
+
+The solution was to recompile GNU-EFI with the -fno-stack-protector GCC
+flag. In GNU-EFI, this can be added to the CFLAGS line in Make.defaults.

 
 
 Installing rEFInd
 =================
 
 
 
 Installing rEFInd
 =================
 

-With rEFInd compiled, you can install it. On a UEFI-based system, you'll
-want to copy files on the ESP as follows:
+With rEFInd compiled, you can install it. The easiest way to do this is
+with the refind-install script, which works on both Linux and Mac OS X.
+Alternatively, you can type "make install" to install using this script.
+Note that this script copies files to the ESP and uses "efibootmgr" (on
+Linux) or "bless" (on OS X) to add rEFInd to the firmware's boot loader
+list. The docs/man/refind-install.8 file (and its HTML conversion,
+docs/refind/refind-install.html) provides more details on this script and
+its use.
+
+If refind-install doesn't work for you or if you prefer to do the job
+manually, you may. On a UEFI-based system, you'll want to copy files on the
+ESP as follows:

 
 * Create a directory for rEFInd, such as EFI/refind.
 
 * Create a directory for rEFInd, such as EFI/refind.

-* Copy refind/refind.efi to the ESP's EFI/refind directory.
+* Copy refind/refind_ia32.efi or refind_x64.efi to the ESP's EFI/refind
+  directory.

 * Copy refind.conf-sample to the EFI/refind directory as refind.conf.
 * Copy the icons subdirectory, including all its files, to EFI/refind.
 
 You'll then need to activate rEFInd in your EFI. This can be done with
 tools such as "efibootmgr" under Linux or "bless" under OS X. See the
 * Copy refind.conf-sample to the EFI/refind directory as refind.conf.
 * Copy the icons subdirectory, including all its files, to EFI/refind.
 
 You'll then need to activate rEFInd in your EFI. This can be done with
 tools such as "efibootmgr" under Linux or "bless" under OS X. See the

-docs/installing.html file for details.
+docs/refind/installing.html file for details.
+
+
+Note to Distribution Maintainers
+================================
+
+The refind-install script, and therefore the "install" target in the
+Makefile, installs the program directly to the ESP and it modifies the
+*CURRENT COMPUTER's* NVRAM. Thus, you should *NOT* use this target as part
+of the build process for your binary packages (RPMs, Debian packages,
+etc.). (Gentoo could use it in an ebuild, though....) You COULD, however,
+install the files to a directory somewhere (/usr/share/refind or whatever)
+and then call refind-install as part of the binary package installation
+process. Placing the files directly in /boot/efi/EFI/{distname}/refind and
+then having a post-install script call efibootmgr is probably the better
+way to go, but this assumes that the ESP is mounted at /boot/efi.
+
+
+Compiling the EFI Filesystem Drivers
+====================================
+
+To build all the drivers, you can type "make fs", "make fs_tiano", or "make
+fs_gnuefi" from the main directory, which builds the drivers and places
+copies in both the filesystems and drivers_{arch} subdirectories.
+
+To install drivers, you can type "make install" in the "filesystems"
+directory. This copies all the drivers to the
+"/boot/efi/EFI/refind/drivers" directory. Alternatively, you can copy the
+files you want manually. The refind-install script includes an optional
+"--drivers" option that will install the drivers along with the main rEFInd
+program, but to the drivers_{arch} subdirectory of the main rEFInd
+installation directory.
+
+*CAUTION:* Install drivers for your system's architecture *ONLY*.
+Installing drivers for the wrong architecture causes some systems to hang
+at boot time. This risk can be minimized by including the architecture code
+in the drivers subdirectory name (drivers_x64 or drivers_ia32).
+
+The drivers all rely on filesystem wrapper code created by rEFIt's author,
+Christoph Pfisterer. Most of the drivers seem to have passed through
+Oracle's VirtualBox project (https://www.virtualbox.org) and the Clover
+boot loader project (https://sourceforge.net/projects/cloverefiboot/),
+which I used as the source for this build.
+
+Adding Support for Network Boot
+===============================
+
+rEFInd provides EXPERIMENTAL support for booting over the network using
+iPXE (http://ipxe.org) as a means to receive the payload. In order to
+enable this feature you'll want to follow these instructions:
+
+* cd net/
+* make source
+* make netboot
+* copy bin/ipxe.efi and bin/ipxe_discover.efi to the EFI volume at EFI/tools/
+
+Note that you may need to install additional development packages, such as
+libiberty-dev and binutils-dev, in addition to those needed to build rEFInd
+itself.
+
+My own tests show this support to work under optimal conditions; however,
+architecture (EFI vs. BIOS) detection may not work, and some computers will
+hang or won't retrieve boot files from the network. For these reasons, this
+support is disabled by default in rEFInd, and I do not provide iPXE
+binaries.