NOTE! This site uses cookies and similar technologies.

If you not change browser settings, you agree to it.

I understand

Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1
  • 2

TOPIC: Out of kernel modules in initramfs

Out of kernel modules in initramfs 3 years 1 month ago #1

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
Hi guys.

I'm adding ZFS support packages to OpenELEC (actually this should be in past tense :lol:).
To be able to correctly mount the /storage as ZFS i need to do this in initramfs, so i've added the _init() stuff into the zfs package.mk, added it as dependency to initramfs, modified the busybox init script to be able to mount zfs pool from kernel cmdline (instead of disk=LABEL=Storage you pass disk=ZFS=poolname,options - kinda like for iscsi), added few additional libs to initramfs (from util-linux and glibc).

Everything works flawlessly, but...
To compile the zfs (and spl) modules i need to have "installed" kernel sources (the linux package) but to compile kernel sources i need to have "installed" initramfs package, so automagic make release/image doesn't work... (let's say that is an circular dependency)

Right now, make this work, i need to:
  1. comment out the "zfs:init" dependency in initramfs
  2. install the kernel (scripts/install kernel)
  3. remove the install_target kernel stamp (without scripts/clean linux of course)
  4. uncomment the "zfs:init" dependency
  5. do make release/image or scripts/install linux (tl;dr continue the build process)

So, the question is: Does anybody have an idea how to make this automagic?
Last Edit: 3 years 1 month ago by jabbas.
The administrator has disabled public write access.
The following user(s) said Thank You: mantis3dfx

Out of kernel modules in initramfs 3 years 1 month ago #2

  • lrusak
  • lrusak's Avatar
You shouldn't even need to change the initramfs package.

Just use the testing package. In packages/testing/package.mk, add
$SCRIPTS/install zfs

Then in the project options file set testting to "yes"
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #3

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
lrusak wrote:
You shouldn't even need to change the initramfs package.

Just use the testing package. In packages/testing/package.mk, add
<code>$SCRIPTS/install zfs</code>

Then in the project options file set testting to "yes"

Hmm, according to the description in testing/package.mk this is used to "testing: Metapackage for various packages to test while developing". So for example if i need a strace program in the OpenELEC image to debug something etc...

Still, even then, this will not work for me, due to the fact, that i need to append zpool and zfs commands into initramfs.cpio (built by initramfs package) and to do that i need to have built kernel package... Building kernel package need finished initramfs package (initramfs.cpio file to be exact), so we have a never ending loop here...

In big distros this is done by splitting the kernel and initramfs into separate files (not embedding the initramfs into kernel) and passing initramfs= param into syslinux/grub/whatever. I'm not saying that embedding the initramfs into kernel is bad idea, it's perfect (especially for embedded machines).

Thing is that i need to compile zfs between 'make' and 'make install' phase of kernel building process.
Last Edit: 3 years 1 month ago by jabbas.
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #4

  • lrusak
  • lrusak's Avatar
Can you share you package.mk for zfs? I'd like to take a look.
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #5

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
Here you go:

spl/package.mk (this module is required by zfs - spl = solaris porting layer)
################################################################################
#      This file is part of OpenELEC - http://www.openelec.tv
#      Copyright (C) 2009-2014 Stephan Raue (stephan@openelec.tv)
#
#  OpenELEC is free software: you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation, either version 2 of the License, or
#  (at your option) any later version.
#
#  OpenELEC is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with OpenELEC.  If not, see <http://www.gnu.org/licenses/>.
################################################################################

PKG_NAME="spl"
PKG_VERSION="0.6.3"
PKG_REV="1"
PKG_ARCH="any"
PKG_LICENSE="GPLv2"
PKG_SITE="http://zfsonlinux.org/"
PKG_URL="http://archive.zfsonlinux.org/downloads/zfsonlinux/${PKG_NAME}/${PKG_NAME}-${PKG_VERSION}.tar.gz"
#PKG_DEPENDS_TARGET="linux"
#PKG_DEPENDS_INIT="linux"
PKG_NEED_UNPACK="$LINUX_DEPENDS"
PKG_PRIORITY="optional"
PKG_SECTION="driver"
PKG_SHORTDESC="zfs"
PKG_LONGDESC="zfs"

PKG_IS_ADDON="no"
PKG_AUTORECONF="no"

PKG_CONFIGURE_OPTS_TARGET="--with-linux=$(kernel_path) \
						   --with-linux-obj=$(kernel_path)"

PKG_CONFIGURE_OPTS_INIT="${PKG_CONFIGURE_OPTS_TARGET}
						 --disable-shared
						 --enable-static"

post_makeinstall_target() {
  rsync -a $INSTALL/usr/src/${PKG_NAME}-${PKG_VERSION} $SYSROOT_PREFIX/usr/src/${PKG_NAME}-${PKG_VERSION}
  rm -rf $INSTALL/usr/src/${PKG_NAME}-${PKG_VERSION}
}

makeinstall_init() {
  mkdir -p ${INSTALL}/etc ${INSTALL}/lib/modules
  for mod in $(find module -name \*.ko); do
    cp $mod ${INSTALL}/lib/modules/
  done
  echo spl > ${INSTALL}/etc/modules-${PKG_NAME}
}

zfs/package.mk
################################################################################
#      This file is part of OpenELEC - http://www.openelec.tv
#      Copyright (C) 2009-2014 Stephan Raue (stephan@openelec.tv)
#
#  OpenELEC is free software: you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation, either version 2 of the License, or
#  (at your option) any later version.
#
#  OpenELEC is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with OpenELEC.  If not, see <http://www.gnu.org/licenses/>.
################################################################################

PKG_NAME="zfs"
PKG_VERSION="0.6.3"
PKG_REV="1"
PKG_ARCH="any"
PKG_LICENSE="CDDL"
PKG_SITE="http://zfsonlinux.org/"
PKG_URL="http://archive.zfsonlinux.org/downloads/zfsonlinux/${PKG_NAME}/${PKG_NAME}-${PKG_VERSION}.tar.gz"
PKG_DEPENDS_TARGET="spl zlib util-linux"
PKG_DEPENDS_INIT="spl zlib:init util-linux:init"
PKG_NEED_UNPACK="$LINUX_DEPENDS"
PKG_PRIORITY="optional"
PKG_SECTION="driver"
PKG_SHORTDESC="zfs"
PKG_LONGDESC="zfs"

PKG_IS_ADDON="no"
PKG_AUTORECONF="no"

# TODO/FIXME there are some problems with blkid while ./configure
PKG_CONFIGURE_OPTS_TARGET="--with-blkid=no \
						   --with-linux=$(kernel_path) \
						   --with-linux-obj=$(kernel_path) \
						   --with-spl=${SYSROOT_PREFIX}/usr/src/spl-${PKG_VERSION} \
						   --with-spl-obj=${SYSROOT_PREFIX}/usr/src/spl-${PKG_VERSION}/$(get_module_dir)"

PKG_CONFIGURE_OPTS_INIT="${PKG_CONFIGURE_OPTS_TARGET}
						 --disable-shared
						 --enable-static"

post_makeinstall_target() {
  rsync -a $INSTALL/usr/src/${PKG_NAME}-${PKG_VERSION} $SYSROOT_PREFIX/usr/src/${PKG_NAME}-${PKG_VERSION}
  rm -rf $INSTALL/usr/src/${PKG_NAME}-${PKG_VERSION}
}

makeinstall_init() {
  mkdir -p ${INSTALL}/bin ${INSTALL}/sbin ${INSTALL}/etc ${INSTALL}/lib/modules
  cp cmd/zpool/zpool ${INSTALL}/sbin
  cp cmd/zfs/zfs ${INSTALL}/sbin
  cp cmd/mount_zfs/mount.zfs ${INSTALL}/sbin

  for mod in $(find module -name \*.ko); do
    cp $mod ${INSTALL}/lib/modules/
  done

  for mod in zunicode zavl znvpair zcommon zfs; do
    echo $mod >> ${INSTALL}/etc/modules-${PKG_NAME}
    echo $mod ${INSTALL}/etc/modules-${PKG_NAME}
  done

}

P.S.:
Here's my fork on github: github.com/jabbas/OpenELEC.tv
Last Edit: 3 years 1 month ago by jabbas.
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #6

  • ultraman
  • ultraman's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 4647
  • Thank you received: 889
  • Karma: 324
I think both this packages needs to be build as :init. And depends set in initramfs package which would pack everything.
Will try later when build one x86_64 image.
ultraman == vpeter
LAMP addon - Linux, Apache, MySQL PHP bundle
Sundtek and HDHomeRun DVB driver addons
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #7

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
ultraman wrote:
I think both this packages needs to be build as :init. And depends set in initramfs package which would pack everything.
Will try later when build one x86_64 image.

And they are, the only problem here is that spl and zfs requires the built kernel to compile it. But to build the kernel we need initramfs to be installed (to append the initramfs.cpio into the kernel) and initramfs needs to have built zfs which needs kernel...

So: kernel needs initramfs which needs zfs which needs kernel.

I need to split building/installing kernel into two parts compiling and 'make bzImage' and build the initramfs between.
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #8

  • ultraman
  • ultraman's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 4647
  • Thank you received: 889
  • Karma: 324
Now I see the problem - it's the same as chicken or the egg ;)

I would try this: in initramfs package I would add dependency for zfs. In zfs package make_init() I would call scripts/build linux.
ultraman == vpeter
LAMP addon - Linux, Apache, MySQL PHP bundle
Sundtek and HDHomeRun DVB driver addons
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #9

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
ultraman wrote:
I would try this: in initramfs package I would add dependency for zfs. In zfs package make_init() I would call scripts/build linux.

It will work, probably, but doesn't look kosher to me :P

What about splitting linux package into two:
  1. linux - this one will build the sources, install them like now (but without depending on initramfs)
  2. linux-image - this one will do 'make bzImage' and stuff' (with dependening on initramfs)
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #10

  • ultraman
  • ultraman's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 4647
  • Thank you received: 889
  • Karma: 324
Still building my image. Will let you know if I got something useful.
ultraman == vpeter
LAMP addon - Linux, Apache, MySQL PHP bundle
Sundtek and HDHomeRun DVB driver addons
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #11

  • sraue
  • sraue's Avatar
  • Offline
  • Developer
  • Posts: 2094
  • Thank you received: 650
  • Karma: 131
is it not enough to todo a "make modules" before building the zfs modules? the steps we do:
make modules
./scripts/install initramfs
make bzImage

so adding zfs:init in initramfs/package.mk should work?
or whats the error you get? a module should not depend on make bzImage
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #12

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
is it not enough to todo a "make modules" before building the zfs modules?[/quote]
I'm not sure. I'll download the vanilla kernel and will try to compile zfs with it and i'll let you know.
sraue wrote:
the steps we do:
make modules
./scripts/install initramfs
make bzImage

so adding zfs:init in initramfs/package.mk should work?
or whats the error you get? a module should not depend on make bzImage

zfs:init is there :)

Don't remember the exact error message, but i found on zfsonlinux that this is due to the fact that kernel is not compiled.
As far as i know, compiling out-of-kernel modules should only need 'make config' without building so...

As i said, i'll download the vanilla kernel and vanilla zfs packages and i will try to reproduce the error without OpenELEC build system, so i'll get the exact point of failure.
Stay tuned :)
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #13

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
ultraman wrote:
Still building my image. Will let you know if I got something useful.
When building linux will fail (building spl actually) just comment out the spl:init and zfs:init in initramfs/package.mk build linux package and after that do:
rm build.OpenELEC-Generic.x86_64-devel/.stamps/initramfs/build_target
rm build.OpenELEC-Generic.x86_64-devel/image/.stamps/initramfs/install_target
rm build.OpenELEC-Generic.x86_64-devel/image/.stamps/linux/install_target

and do
scripts/build linux
scripts/install linux

this will do the job :)

Anyway, maybe i'm just setting wrong path to kernel in zfs/spl package.mk... This path exists only when kernel is installed...
Last Edit: 3 years 1 month ago by jabbas.
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #14

  • seo
  • seo's Avatar
  • Offline
  • Gold Boarder
  • Posts: 609
  • Thank you received: 168
  • Karma: 39
in packages/linux/package.mk
  ( cd $ROOT
    $SCRIPTS/install initramfs
  )

  LDFLAGS="" make bzImage

should be moved to make_init() { }

EDIT: with a bit of rework, of course.
Last Edit: 3 years 1 month ago by seo.
The administrator has disabled public write access.

Out of kernel modules in initramfs 3 years 1 month ago #15

  • jabbas
  • jabbas's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 13
  • Thank you received: 2
  • Karma: 0
seo wrote:
in packages/linux/package.mk
  ( cd $ROOT
    $SCRIPTS/install initramfs
  )

  LDFLAGS="" make bzImage

should be moved to make_init() { }

EDIT: with a bit of rework, of course.

I have to disagree... make_init() in linux package is used by initramfs to pull the modules from kernel into the initramfs prior to compiling the kernel itself.

It goes like this: build linux runs build initramfs which runs build linux:init

But you inspired me :) If we will build the whole kernel in make_init() and after that we will reuse them in make_target() will do the job for me and zfs. make_target() will reuse compiled tree and just will make bzImage and make initramfs. Therefore, i can depend on linux:init in zfs/spl and everything should build/install cleanly!
The administrator has disabled public write access.
  • Page:
  • 1
  • 2