new package: cvmfs #27139
new package: cvmfs #27139
Conversation
| @@ -1,5 +1,7 @@ | ||
| #!/bin/sh | ||
|
|
||
| +echo "64compact" > arch |
There was a problem hiding this comment.
The default value was giving errors atleast on arm64, with this it worked on both arm64 and x86_64.
Not sure if this could be generalized
There was a problem hiding this comment.
That isn't gonna work for the 32 bit architectures.
But I don't have good answer for what you'd do about it.
${TERMUX_ARCH_BITS} should be available in the environment during buildtime, so that could be used here.
That will be either 64 or 32 depending on the bitness of the architecture.
| @@ -0,0 +1,57 @@ | |||
| diff --git a/cmake/Modules/Findleveldb.cmake b/cmake/Modules/Findleveldb.cmake | |||
There was a problem hiding this comment.
Hard setting the values to the one internally compiled.
Was having trouble getting find_package not find the .so that would come with the leveldb package.
Besides the internal patches to this library by CVMFS, i was also having trouble due to the name expected by libcvmfs_fuse for leveldb to be mangled while the one the upstream libleveldb.so are not (so something was going wrong there)
There was a problem hiding this comment.
This should be settable by flag, e.g. -DLEVELDB_LIB_DIR=$TERMUX_PREFIX/lib/leveldb, although I'm not 100% sure that path is correct.
| @@ -0,0 +1,13 @@ | |||
| diff --git a/doc/manpages/CMakeLists.txt b/doc/manpages/CMakeLists.txt | |||
There was a problem hiding this comment.
When cross-compiling help2man cannot use the executables produced to generate the docs.
Could add a check here to do this step in case we are not cross compiling if we really want the man page
There was a problem hiding this comment.
That's a known issue with packages using help2man.
- You might be able to use the same trick I used for
lowdownin thebtopbuild in
addpkg(main/btop): 1.4.5 #27110
Otherwise, disabling the man pages is fine, we prefer to ship manuals, but if it's not doable when cross compiling that is fine.
TomJo2000
left a comment
TomJo2000
left a comment
There was a problem hiding this comment.
You've clearly put in a ton of work into this package.
Thank you for taking the time to put up with our build system.
I can't seem to find a package request for this project.
So I'll just go through an abbreviated version of the package request checklist below really quick.
Note
For future reference, please consider opening a package request before spending the time to write a package build.
Package requests give us a heads up to evaluate if a package is suitable for packaging in the main Termux repositories.
https://github.com/termux/termux-packages/issues/new?template=02-package-request.yml
If you intend to package
Not all projects will meet out packaging guidelines.
In case this project does not, it may still be suitable for inclusion in the TUR
Packaging policy acknowledgement
-
The project is actively developed.
-
[~] The project has existing packages and is "well known".
-
Licensed under an open source license.
-
Not available through a language package manager: pip, npm, cpan, cargo, etc.
-
[?] Not taking up too much disk space (< 100MiB per architecture, exceptions can be made)
-
Not duplicating the functionality of existing packages.
-
Not serving hacking, malware, phishing, spamming, spying, ddos functionality.
-
I certify that I have read Termux Packaging Policy and understand that my request will be denied if it is found lacking.
I'm not exactly sure what the usecase for this package would be on Termux, and this may be a better fit for the TUR, but I'm not sure if and how the TUR does root packages.
Please do feel free to continue to use this PR for build testing in the meantime.
| @@ -1,5 +1,7 @@ | ||
| #!/bin/sh | ||
|
|
||
| +echo "64compact" > arch |
There was a problem hiding this comment.
That isn't gonna work for the 32 bit architectures.
But I don't have good answer for what you'd do about it.
${TERMUX_ARCH_BITS} should be available in the environment during buildtime, so that could be used here.
That will be either 64 or 32 depending on the bitness of the architecture.
| @@ -0,0 +1,57 @@ | |||
| diff --git a/cmake/Modules/Findleveldb.cmake b/cmake/Modules/Findleveldb.cmake | |||
There was a problem hiding this comment.
This should be settable by flag, e.g. -DLEVELDB_LIB_DIR=$TERMUX_PREFIX/lib/leveldb, although I'm not 100% sure that path is correct.
| @@ -0,0 +1,13 @@ | |||
| diff --git a/doc/manpages/CMakeLists.txt b/doc/manpages/CMakeLists.txt | |||
There was a problem hiding this comment.
That's a known issue with packages using help2man.
- You might be able to use the same trick I used for
lowdownin thebtopbuild in
addpkg(main/btop): 1.4.5 #27110
Otherwise, disabling the man pages is fine, we prefer to ship manuals, but if it's not doable when cross compiling that is fine.
root-packages/cvmfs/build.sh
Outdated
| export TERMUX_HOST_PLATFORM | ||
| export TERMUX_PKG_API_LEVEL | ||
| export CLANG_TARGET_TRIPLE="${TERMUX_HOST_PLATFORM}${TERMUX_PKG_API_LEVEL}" | ||
| export TERMUX_STANDALONE_TOOLCHAIN | ||
| export LDFLAGS |
There was a problem hiding this comment.
The $TERMUX_* variables should already be exported.
There was a problem hiding this comment.
I think we need atleast TERMUX_STANDALONE_TOOLCHAIN as the --sysroot=${TERMUX_STANDALONE_TOOLCHAIN}/sysroot in 0002-crosscompile-pacparser.patch results in just /sysroot otherwise
I removed the other 2 as they also should not be used anymore after i implemented CLANG_TARGET_TRIPLE
|
I think we can definitely consider this package for a packaging policy exception under the "scientific software" (e.g. |
|
Thanks for looking into this this quick! Sorry for not opening a package request beforehand should've definitely done that.
That would be nice. Also was keeping this as a draft to se if the CVMFS developers would like to chime in on this PR. |
|
cvmfs isn't something I'd ever heard of before, but such it is with a lot of scientific software I support. I don't think it should be dismissed out of hand just because it doesn't neatly fit within our standard relevancy criteria. If someone from upstream wants to chime in packaging feedback is always appreciated and we try not to ship packages in a way their upstream has disapproved of. |
Crivella
force-pushed
the
feature-cvmfs
branch
from
718b4fe to
4d12684
Compare
|
The builds on 32 bit systems seems to both be hitting the problem i described as a potential issue
I am really unsure on how to solve that as the sizeofs in the generated headers ends up not matching the checks in the pre-compiler asserts |
I am going to debug that next and I will let you know if I know how to solve it or not |
Basically the external source of The default of EDIT: Was able to test |
This changeset appears to enable 32-bit support in your package. Please let me know if you have any questions about it. diff --git a/root-packages/cvmfs/0032-pass-socklen_t-to-accept.patch b/root-packages/cvmfs/0032-pass-socklen_t-to-accept.patch
new file mode 100644
index 0000000000..f54e922d7d
--- /dev/null
+++ b/root-packages/cvmfs/0032-pass-socklen_t-to-accept.patch
@@ -0,0 +1,17 @@
+Fixes error
+mount/mount.cvmfs.cc:222:24: error: no matching function for call to 'accept'
+note: candidate function not viable: no known conversion from 'unsigned int *'
+to 'socklen_t * _Nullable' (aka 'int *') for 3rd argument
+when building for 32-bit Android
+
+--- a/mount/mount.cvmfs.cc
++++ b/mount/mount.cvmfs.cc
+@@ -218,7 +218,7 @@ static int GetExistingFuseFd(const string &fqrn, const string &workspace,
+ int fuse_fd = -1;
+ if (result == "OK") {
+ struct sockaddr_un addr;
+- unsigned int len = sizeof(addr);
++ socklen_t len = sizeof(addr);
+ const int con_fd = accept(recv_sock_fd,
+ reinterpret_cast<struct sockaddr *>(&addr), &len);
+ fuse_fd = RecvFdFromSocket(con_fd);
diff --git a/root-packages/cvmfs/build.sh b/root-packages/cvmfs/build.sh
index 62694a779b..2633c7fffa 100644
--- a/root-packages/cvmfs/build.sh
+++ b/root-packages/cvmfs/build.sh
@@ -32,16 +32,16 @@ TERMUX_PKG_EXTRA_CONFIGURE_ARGS="
-DLibArchive_LIBRARY=$TERMUX_PREFIX/lib/libarchive.so
-DSPARSEHASH_INCLUDE_DIR=$TERMUX_PREFIX/include/sparsehash
--DLEVELDB_INCLUDE_DIR=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/include
--DLEVELDB_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/lib/libleveldb.a
--DPACPARSER_INCLUDE_DIR=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/include
--DPACPARSER_LIBRARY=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/lib/libpacparser.a
--DVJSON_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/include
--DVJSON_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/lib/libvjson.a
--DSHA3_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/include
--DSHA3_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/lib/libsha3.a
--DLibcrypto_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/crypto/include
--DLibcrypto_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/crypto/lib/libcrypto.a
+-DLEVELDB_INCLUDE_DIR=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/include
+-DLEVELDB_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/lib/libleveldb.a
+-DPACPARSER_INCLUDE_DIR=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/include
+-DPACPARSER_LIBRARY=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/lib/libpacparser.a
+-DVJSON_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/include
+-DVJSON_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/lib/libvjson.a
+-DSHA3_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/include
+-DSHA3_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/lib/libsha3.a
+-DLibcrypto_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/crypto/include
+-DLibcrypto_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH/arm/armv7-a/}/crypto/lib/libcrypto.a
"
termux_step_pre_configure () {
@@ -57,4 +57,10 @@ termux_step_pre_configure () {
export CLANG_TARGET_TRIPLE="${TERMUX_HOST_PLATFORM}${TERMUX_PKG_API_LEVEL}"
export TERMUX_STANDALONE_TOOLCHAIN
export LDFLAGS
+
+ patch="$TERMUX_PKG_BUILDER_DIR/crosscompile-pacparser.diff"
+ echo "Applying patch: $(basename "$patch")"
+ test -f "$patch" && sed \
+ -e "s%\@TERMUX_ARCH_BITS\@%${TERMUX_ARCH_BITS}%g" \
+ "$patch" | patch --silent -p1
}
diff --git a/root-packages/cvmfs/0002-crosscompile-pacparser.patch b/root-packages/cvmfs/crosscompile-pacparser.diff
similarity index 96%
rename from root-packages/cvmfs/0002-crosscompile-pacparser.patch
rename to root-packages/cvmfs/crosscompile-pacparser.diff
index dc725a5a33..802c5304e0 100644
--- a/root-packages/cvmfs/0002-crosscompile-pacparser.patch
+++ b/root-packages/cvmfs/crosscompile-pacparser.diff
@@ -77,7 +77,7 @@ index 55b1a95c5..3acb323d2 100755
[ -d $static_result_dir ] && rm -fR $static_result_dir
make $FIX_PYTHON -C src clean -j ${CVMFS_BUILD_EXTERNAL_NJOBS}
-make $FIX_PYTHON $FIX_COMP CVMFS_BASE_C_FLAGS="$CVMFS_BASE_C_FLAGS" -j1 -C src pacparser.o spidermonkey/libjs.a # default target runs tests!
-+make $FIX_PYTHON $FIX_COMP CVMFS_BASE_C_FLAGS="$CVMFS_BASE_C_FLAGS" CC="clang" -j1 -C src pacparser.o spidermonkey/libjs.a # default target runs tests!
++make $FIX_PYTHON $FIX_COMP CVMFS_BASE_C_FLAGS="$CVMFS_BASE_C_FLAGS" CC="clang -m@TERMUX_ARCH_BITS@" -j1 -C src pacparser.o spidermonkey/libjs.a # default target runs tests!
echo "finished internal build of libpacparser"
echo "creating static link library for libpacparser..."I have not tested it at runtime on 32-bit devices beyond very basic trivial invocations, like this. Would you like to test it yourself (which is possible to do on some phones that are otherwise 64-bit by installing the 32-bit ARM build of Termux APK), or would you like me to try to test it? If you would like me to try to test it on one of my 32-bit devices, would it be not too much trouble for you to write some test scripts I could run that you think would effectively test the functionality of the package on 32-bit devices? |
Thank you very much. Will try to test it, but also do not mind adding some test scripts |
|
@robertkirkman Again thanks for the help on the 32 bit it is compiling and working now 
I've been using eessi for testing as this is the platform i work on, but we can also use so you wont need the extra public key found in This is the edited config file i use and you can mount it whit this set of commands (starting from home) 
|
root-packages/cvmfs/build.sh
Outdated
| -DSHA3_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/include | ||
| -DSHA3_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/lib/libsha3.a | ||
| -DLibcrypto_INCLUDE_DIRS=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/crypto/include | ||
| -DLibcrypto_LIBRARIES=$TERMUX_PKG_SRCDIR/externals_install.${TERMUX_ARCH}/crypto/lib/libcrypto.a |
There was a problem hiding this comment.
When applying the 32-bit changes, you missed applying some of the changes I made in this area, sorry if you already noticed that and are already fixing it, I just thought to mention it just in case you didn't notice
There was a problem hiding this comment.
Thanks i had seen it but had not tested yet on the arm target (only i686).
I've added the string replacements and tested the build locally.
On a sidenote do you know if it is possible to emulate arm on an x86 machine?
I am fairly new to android development and i am using android studio to launch the virtual devices, but it seems the QEMU it ships with does not support cross-architecture emulation (atleast not with a basic install)
There was a problem hiding this comment.
On a sidenote do you know if it is possible to emulate arm on an x86 machine? I am fairly new to android development and i am using
android studioto launch the virtual devices, but it seems the QEMU it ships with does not support cross-architecture emulation (atleast not with a basic install)
Yes it is possible, and there are several ways to do it, but the short explanation is that it's difficult and it requires some study of topics that are more advanced than the default Android Studio emulator. That one does not support ARM on x86 machine.
I have created a complete guide for how to emulate ARM Termux on an x86 device in my favorite way (which is not the only way, but it is, in my opinion, a good balance between accuracy and speed). It is here and it is the script named install-termux-docker-inside-qemu-full-system-emulator.sh.
Just running the script directly on a GNU/Linux 64-bit x86 PC should automatically install everything and start the emulator, as long as you installed the necessary dependencies beforehand which are, for example on Debian and Ubuntu, just sudo apt build-dep qemu-system-x86 openssh-client.
I was kind of wondering "why does it need to have so, I set Here is what I made so far, which makes it completely build with patch
|
Mainly because I was mostly interested in the client functionalities, but also to deal with the problems with one branch of the build at a time. |
yes that's ok, one reason I thought to try it is because you mentioned that you used a If it's too difficult to make work correctly then it can stay disabled.
In my opinion, based on the precedent set by many previous packages, if the package has major functionality that works without root, but other functionality that requires root, the whole package should be in the The reason I have been following that practice is because, if I were to find packages that have some functionality that requires root and start moving them into the |
|
The Anyway might give it a try especially if we can get the upstream dev interested, we are in contact on the EESSI slack |
4 participants
WIP
Adding the cvmfs client to termux to allow mounting CVMFS remote filesystems.
Currently only feature the client part of it.
This would allow access to a vast variety of repositories with data and/or software stacks
Main drive for this (from my side) is to get EESSI to work on android.
Currently able to mount it and tha from a
termux-chrootenvironment run a few of the demos from https://github.com/EESSI/eessi-demoPotential issues: