Opened 7 years ago

Last modified 3 years ago

#40840 new update

s3fs: Update to 1.73 and switch to OSXFuse

Reported by: genecrucean@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version:
Keywords: haspatch Cc: petrrr, mp@…, cooljeanius (Eric Gallager), vulcan_@…, jeremy@…, macports@…, mrdanbrooks (dan brooks)
Port: s3fs

Description (last modified by ryandesign (Ryan Schmidt))

There is a much newer version of s3fs available, especially with the release of Mavericks today. This port seems to have fallen to the side for some reason.

1.61 release: Aug 30, 2011
1.73 release: Aug 16, 2013

Attachments (1)

s3fs_update (2.3 KB) - added by huets (Sylvain Huet) 4 years ago.
diff for s3fs update to 1.82, tested on macOS Sierra clang-802.0.38

Download all attachments as: .zip

Change History (19)

comment:1 in reply to:  description Changed 7 years ago by ryandesign (Ryan Schmidt)

Description: modified (diff)
Keywords: haspatch added
Port: s3fs added
Summary: Update s3fs from 1.61 to 1.73s3fs: Update to 1.73

Replying to genecrucean@…:

This port seems to have fallen to the side for some reason.

Because nobody has volunteered to maintain it.

I took a quick look. On Leopard, 1.61 builds but 1.73 does not. On Mavericks, neither 1.61 nor 1.73 build. So I'm not updating the port today.

comment:2 Changed 7 years ago by petrrr

Cc: Peter.Danecek@… added

Cc Me!

comment:3 Changed 7 years ago by mp@…

Trying to build gives

Requested 'fuse >= 2.8.4' but version of fuse is 2.7.3

Seems like OSXfuse is a bit behind the main FUSE project at http://fuse.sourceforge.net/ , which is at version 2.9.3.

Tried to change the configuration to allow 2.7.3, but then the build very soon fails with

s3fs.cpp:3214:28: error: no member named 'capable' in 'fuse_conn_info'
  if ( (unsigned int)conn->capable & FUSE_CAP_ATOMIC_O_TRUNC) {
                     ~~~~  ^
s3fs.cpp:3214:38: error: use of undeclared identifier 'FUSE_CAP_ATOMIC_O_TRUNC'
  if ( (unsigned int)conn->capable & FUSE_CAP_ATOMIC_O_TRUNC) {
                                     ^
s3fs.cpp:3216:12: error: no member named 'want' in 'fuse_conn_info'
     conn->want |= FUSE_CAP_ATOMIC_O_TRUNC;
     ~~~~  ^
s3fs.cpp:3216:20: error: use of undeclared identifier 'FUSE_CAP_ATOMIC_O_TRUNC'
     conn->want |= FUSE_CAP_ATOMIC_O_TRUNC;
                   ^
4 errors generated.

so it seems to really need for OSXfuse to catch up with the main FUSE.

comment:4 Changed 7 years ago by mp@…

Cc: mp@… added

Cc Me!

comment:5 Changed 7 years ago by mf2k (Frank Schima)

Summary: s3fs: Update to 1.73s3fs: Update to 1.73 and switch to OSXFuse
Version: 2.2.0

This port needs to also switch to osxfuse (see #39456).

I see the same build error with osxfuse.

comment:6 Changed 7 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Cc Me!

comment:7 Changed 7 years ago by mf2k (Frank Schima)

Cc: vulcan_@… added

Cc reporter of dup #43905.

comment:8 Changed 7 years ago by mf2k (Frank Schima)

Cc: jeremy@… added

Cc reporter of dup #44568.

comment:9 Changed 5 years ago by macports@…

Cc: macports@… added

Cc Me!

comment:10 Changed 4 years ago by cbarrett (Colin Barrett)

This is the situation today:

> sudo port install s3fs
--->  Computing dependencies for s3fs
Error: Dependency 'fuse4x' not found.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port s3fs failed

Perhaps the port should be simply delisted or something until it's fixed? Having a broken port hanging around seems bad.

comment:11 Changed 4 years ago by huets (Sylvain Huet)

I am new to macports debugging and I will try to make a modest contribution to solve this problem.

The compilation of s3fs 1.61 also fails on my machine which has the following configuration:
Apple LLVM version 8.1.0 (clang-802.0.38)
Target: x86_64-apple-darwin16.5.0

Problems:
1) fuse4x not found
2) after installing osxfuse, the previous problem disappear but s3fs does not build. the problem is due to the included map file:

:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/map:478:17: error: no matching function for call to object of type 'const case_insensitive_compare_func'
:info:build         {return static_cast<const _Compare&>(*this)(__x.__cc.first, __y);}
:info:build                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tree:2443:14: note: in instantiation of member function 'std::__1::__map_value_compare<std::__1::basic_string<char>, std::__1::__value_type<std::__1::basic_string<char>, std::__1::basic_string<char> >, case_insensitive_compare_func, true>::operator()' requested here

This included map file version seems unsupported:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/map

s3fs compiles after adding this line at the end of the Portfile, to tell macports to use libstdc++ and not libc++, nevertheless, libstdc++ seems deprecated.

configure.cxx_stdlib {libstdc++}

(I found the instructions to build a local Protfile here:
https://guide.macports.org/chunked/development.local-repositories.html)

I also made the following Portfile to build a newer version of s3f:

# -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4

PortSystem 1.0

name                s3fs
version             1.82
revision            1
categories          fuse

platforms           darwin
maintainers         nomaintainer
description         Amazon S3 filesystem for FUSE
long_description    \
    FUSE-based file system backed by Amazon S3. Mount a bucket as a \
    local file system read/write. Store files/folders natively and \
    transparently.


homepage            https://github.com/s3fs-fuse/s3fs-fuse/archive/
distfiles           v${version}.tar.gz
master_sites        ${homepage}
license             GPL-2
checksums           rmd160  c57169e56e3b5764cc6f88f9389777db877209c1 \
                    sha256  8a40f0b11b558b6d733eeff4fcc025cc416df37b6732001bb0c630f6d6d760dd

depends_build       path:bin/pkg-config:pkgconfig
depends_lib         port:curl \
                    port:libxml2 \
                    path:lib/pkgconfig/fuse.pc:osxfuse \
                    bin:ssh:openssh

post-extract {

  file rename ${workpath}/s3fs-fuse-${version} ${workpath}/s3fs-${version}

  # https://github.com/s3fs-fuse/s3fs-fuse/issues/600
  # https://stackoverflow.com/questions/21903805/how-to-download-a-single-commit-diff-from-github
  # https://owenou.com/ten-things-you-didnt-know-git-and-github-could-do
  system -W ${worksrcpath} "wget https://github.com/s3fs-fuse/s3fs-fuse/commit/0c6a3882a27483f0fd9f5196be7cf3e2836f6654.patch"
  system -W ${worksrcpath} "patch --verbose ./src/cache.cpp ./0c6a3882a27483f0fd9f5196be7cf3e2836f6654.patch"

}

# no configure file, have to execute autogen.sh
use_autoreconf      yes

I hope this will help...

comment:12 Changed 4 years ago by mf2k (Frank Schima)

Thank you. Per the guidelines, please instead attach a unified diff of the Portfile so we can easily see what changes you are proposing.

comment:13 Changed 4 years ago by kencu (Ken)

There is definitely a learning curve to MacPorts. No doubt this works for you on your local machine, but to get it accepted into MacPorts for the general case, a few things would need to be looked at.

  1. You'll need to use the github portgroup, rather than setting the homepage, distfiles, version, and master_sites manually.
  2. Downloading the files via wget is in the end almost certainly going to be a no-go. The github portgroup should take care of that for you too.
  3. You won't be able to force Sierra into using libstdc++ and have it accepted into MacPorts -- have to figure out why it won't compile against libc++ and fix that instead.
  4. revision will likely be removed (0) for a new update
  5. patching is done automatically by MacPorts in a patch phase, so you wouldn't call "patch" directly yourself with a system call.

Please don't be frustrated -- it's just the difference between something that works locally, and something that can be expected to work for everybody in the general case.

comment:14 in reply to:  12 Changed 4 years ago by huets (Sylvain Huet)

Frank,

Here is a diff (I integrated the patch file). Is it better to do a pull request?

Regards,
Sylvain.

diff --git a/fuse/s3fs/Portfile b/fuse/s3fs/Portfile
index 427becf..c40a8ca 100644
--- a/fuse/s3fs/Portfile
+++ b/fuse/s3fs/Portfile
@@ -1,31 +1,38 @@
 # -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4
 
 PortSystem 1.0
 
 name                s3fs
-version             1.61
-revision        1
+version             1.82
+revision            0
 categories          fuse
 
 platforms           darwin
 maintainers         nomaintainer
 description         Amazon S3 filesystem for FUSE
 long_description    \
     FUSE-based file system backed by Amazon S3. Mount a bucket as a \
     local file system read/write. Store files/folders natively and \
     transparently.
 
-homepage            https://code.google.com/p/s3fs/
 
-master_sites        googlecode:s3fs
+homepage            https://github.com/s3fs-fuse/s3fs-fuse/archive/
+distfiles           v${version}.tar.gz
+master_sites        ${homepage}
 license             GPL-2
-checksums           rmd160  582c193888466341ecf0cc6472dd2a2b5e3eb803 \
-                    sha256  47e6700c9ec65f85cc1eeb004dcf33b4533b3415466107f310fe167f49f35ec5
+checksums           rmd160  c57169e56e3b5764cc6f88f9389777db877209c1 \
+                    sha256  8a40f0b11b558b6d733eeff4fcc025cc416df37b6732001bb0c630f6d6d760dd
 
 depends_build       path:bin/pkg-config:pkgconfig
 depends_lib         port:curl \
                     port:libxml2 \
-                    path:lib/pkgconfig/fuse.pc:fuse4x \
+                    path:lib/pkgconfig/fuse.pc:osxfuse \
                     bin:ssh:openssh
 
-# use_autoreconf      yes
+patchfiles	    patch-src_cache.cpp.diff          
+
+post-extract {
+  file rename ${workpath}/s3fs-fuse-${version} ${workpath}/s3fs-${version}
+}
+
+use_autoreconf      yes
diff --git a/fuse/s3fs/files/patch-src_cache.cpp.diff b/fuse/s3fs/files/patch-src_cache.cpp.diff
new file mode 100644
index 0000000..1cc7907
--- /dev/null
+++ b/fuse/s3fs/files/patch-src_cache.cpp.diff
@@ -0,0 +1,11 @@
+--- src/cache.cpp
++++ src/cache.cpp.new
+@@ -59,7 +59,7 @@ using namespace std;
+ #ifdef HAVE_CLOCK_GETTIME
+ static int s3fs_clock_gettime(int clk_id, struct timespec* ts)
+ {
+-  return clock_gettime(clk_id, ts);
++  return clock_gettime(static_cast<clockid_t>(clk_id), ts);
+ }
+ #else
+ static int s3fs_clock_gettime(int clk_id, struct timespec* ts)

Replying to mf2k:

Thank you. Per the guidelines, please instead attach a unified diff of the Portfile so we can easily see what changes you are proposing.

comment:15 Changed 4 years ago by mf2k (Frank Schima)

Thanks. Since the port is on GitHub, please use the Github portgroup which simplifies the portfile.

comment:16 Changed 4 years ago by mf2k (Frank Schima)

Also, attaching the unified diff to the ticket is preferred because Trac has syntax highlighting for diff files.

Changed 4 years ago by huets (Sylvain Huet)

Attachment: s3fs_update added

diff for s3fs update to 1.82, tested on macOS Sierra clang-802.0.38

comment:17 Changed 4 years ago by ryandesign (Ryan Schmidt)

As mentioned previously, please use the github portgroup, observing the documentation for how to use it in the comments at the top of the file. Remember to use whitespace that conforms to the modeline (indentations should be at multiples of 4 spaces).

comment:18 Changed 3 years ago by mf2k (Frank Schima)

Cc: mrdanbrooks added

Has duplicate #55055.

Note: See TracTickets for help on using tickets.