[[PageOutline]] = Current status of setting up the CI system = This page shall describe what has to be done to get a KDE/CI system up and running on OSX. Partially this is achieved by using a few ports from MacPorts. == Start with a fresh Mavericks/MacPorts install == Currently using the official MacPorts version 2.3.1 is recommended. To be on the safe side make sure your disk has about 60G space in total. However, currently a Mavericks 10.9.2 KDE/CI system on an i7-iMac uses something like this: {{{ $ df -h / Filesystem Size Used Avail Use% Mounted on /dev/disk0s2 64G 36G 29G 56% / }}} that is after all frameworks have been built and installed as shown further down. == Getting a MacPorts-compatible bash environment == Install ports for MacPorts' bash {{{ $ sudo port install bash bash-completion }}} and set up the [https://projects.kde.org/projects/playground/sdk/macports-kde/repository/revisions/master/show/contrib/user-setup bash environment for MacPorts use] regarding search paths (PATH, MANPATH) == Apple developer tools == Install * Xcode from AppStore (and agree actively to the developer license agreement) * developer command line tools {{{ $ xcode-select --install }}} == Tools already present on OSX == Some tools are already installed on OSX due to Xcode (like git, svn, ssh, rsync, make, clang etc.) they don't need to be installed via MacPorts for now. Should a more up-to-date version of any one of these be necessary for the CI system it could be added later. == Some more software supplied by MacPorts == Install ports needed for the CI scripts as well as KF5: {{{ $ sudo port install gnutar py27-lxml bazaar mercurial wget automake autoconf cppcheck shared-mime-info docbook-xml docbook-xsl giflib boost }}} == Basic setup of CI system == Create some folders, clone and update CI system (assuming a user "kdeci" with its home directory "/Users/kdeci"): {{{ $ cd $ mkdir -p WC/KDECI-build; $ cd WC $ git clone git://anongit.kde.org/websites/build-kde-org $ cd build-kde-org $ git checkout production $ ./update-setup.sh # Ignore 404 occurring due to error cloning ECMA262 by hg }}} The update shell script created a folder {{{~/scripts}}} in which subsequent commands will have to be executed. == 1st step: Qt5 build == === Prepare environment for Qt5 build === Install additional ports specific for Qt5 as well as KF5, set correct path to KDE-install directory in config/build/darwin-mavericks.cfg: {{{ $ sudo port install zlib openssl dbus jpeg tiff libmng libpng mysql55 pkgconfig sqlite2 $ sudo port uninstall cmake # Remove MacPorts' cmake (which came into the system as build-dependency for mysql55) $ cd ~/scripts $ cat config/build/darwin-mavericks.cfg [DEFAULT] scriptsLocation=/Users/kdeci/scripts opSys=darwin architecture=mavericks compiler=clang [General] installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ $ python2.7 tools/prepare-environment.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/qt5 ... $ }}} === Set up build environment for Qt5 === Make sure all libs can be found by adding search paths and avoid inclusion of MacPorts' glib2: {{{ $ cd ~/scripts; cat config/build/qt5/darwin-mavericks.cfg [Build] configureCommand=%(configureExecutable)s -release -system-zlib -system-libpng -system-libjpeg -system-sqlite -dbus -plugin-sql-mysql -nomake examples -confirm-license -opensource -prefix {instPrefix} -no-framework -v -I/opt/local/include -L/opt/local/lib -no-glib }}} Consider here the use of options "{{{-debug -separate-debug-info}}}" instead of "{{{-release}}}". Include MySQL's binary path into PATH e.g. in {{{.macports/profile}}} (assuming /opt/local as MacPort's prefix): {{{ export PATH=/opt/local/lib/mysql55/bin:$PATH }}} Set up SSH for rsync access to build host, then start building Qt5: {{{ $ cat ~/.ssh/config Host build.kde.org Port 2022 $ ssh-keygen -C YOUR@EMAIL.ADDRESS # <-- send your public SSH key to Ben Cooksley in order to get access to the build server $ python2.7 tools/perform-build.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/qt5 ... }}} == Installation of cmake == Cmake needs a manual checkout of its sources: {{{ $ python2.7 tools/prepare-environment.py --project cmake --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/cmake ... $ git clone git://cmake.org/cmake.git ~/WC/KDECI-build/cmake $ python2.7 tools/perform-build.py --project cmake --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/cmake ... }}} == Keeping build environment up-to-date == For KF5 one has to modify the configuration settings by commenting out the kauth dependency on polkit-qt-1, since that dependency is only needed on Linux: {{{ $ cd ~/scripts $ ./update-setup.sh $ vi config/base/kf5-qt5 $ grep kauth config/base/kf5-qt5 #frameworks/kauth: kdesupport/polkit-qt-1 $ }}} Problems appeared for some frameworks since kconfig_compiler_kf5.app doesn't yet get installed on OSX (the build scripts error out with "/Applications/KDE/kconfig_compiler_kf5.app/Contents/MacOS/kconfig_compiler_kf5 as location is wrong"). This can be fixed for now introducing a temporary configuration file for kconfig: {{{ $ cat ~/scripts/config/build/kconfig/darwin-mavericks.cfg [DEFAULT] configureExtraArgs=-DCMAKE_INSTALL_BUNDLEDIR="lib/libexec/kf5" }}} which makes sure that the application bundle gets installed below PREFIX/lib/libexec/. In order to always have the latest KDE/CI running it makes sense to temporally stash away the changes made locally and upgrade via git {{{ $ # Check what has changed locally: $ git diff diff --git a/config/base/kf5-qt5 b/config/base/kf5-qt5 index 16abdf6..fff9647 100644 --- a/config/base/kf5-qt5 +++ b/config/base/kf5-qt5 @@ -18,7 +18,7 @@ general/vc: -qt5 general/shared-desktop-ontologies: -qt5 # KDE Frameworks -frameworks/kauth: kdesupport/polkit-qt-1 +#frameworks/kauth: kdesupport/polkit-qt-1 kde/*: general/kdesupport-svn kde/workspace/plasma-workspace: general/libdbusmenu-qt diff --git a/config/build/darwin-mavericks.cfg b/config/build/darwin-mavericks.cfg index cb62c49..0648464 100644 --- a/config/build/darwin-mavericks.cfg +++ b/config/build/darwin-mavericks.cfg @@ -1,7 +1,9 @@ [DEFAULT] +scriptsLocation=/Users/kdeci/scripts opSys=darwin architecture=mavericks compiler=clang +configurePlatformArgs=-DCMAKE_INSTALL_BUNDLEDIR="Applications/KF5" -DDATA_INSTALL_DIR="Library/Application Support" -DCMAKE_DISABLE_FIND_PACKAGE_X11=ON [General] -installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ \ No newline at end of file +installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ diff --git a/tools/environment-generator.py b/tools/environment-generator.py index 37dcbae..e46a13c 100644 --- a/tools/environment-generator.py +++ b/tools/environment-generator.py @@ -31,7 +31,7 @@ environment = manager.generate_environment(True) neededVariables = [ 'CMAKE_PREFIX_PATH', 'KDEDIRS', 'PATH', 'LD_LIBRARY_PATH', 'PKG_CONFIG_PATH', 'PYTHONPATH', 'PERL5LIB', 'QT_PLUGIN_PATH', 'QML_IMPORT_PATH', 'QML2_IMPORT_PATH', 'XDG_DATA_DIRS', - 'XDG_CONFIG_DIRS', 'QMAKEFEATURES', 'XDG_CURRENT_DESKTOP' + 'XDG_CONFIG_DIRS', 'QMAKEFEATURES', 'XDG_CURRENT_DESKTOP', 'DYLD_LIBRARY_PATH' ] # Generate the shell format environment file, suitable for sourcing $ $ git status On branch production Your branch is up-to-date with 'origin/production'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: config/base/kf5-qt5 modified: config/build/darwin-mavericks.cfg Untracked files: (use "git add ..." to include in what will be committed) build.sh dependencies/ kapidox/ kde_projects.xml poppler-test-data/ prepare.sh no changes added to commit (use "git add" and/or "git commit -a") $ $ # Updating CI scripts $ cd ~/scripts; git stash; git pull; git stash pop }}} '''Note 1:''' It may be that the DATA_INSTALL_DIR setting is actually not needed! '''Note 2:''' DYLD_LIBRARY_PATH has only temporally been added due to wrong linking of phonon (e.g. for kate and kwrite). == Installation of other projects and KDE frameworks == The various frameworks have to be handled by replacing "PROJECT" with the framework's project name: {{{ $ python2.7 tools/prepare-environment.py --project PROJECT --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/PROJECT ... $ (cd ~/WC/KDECI-build/PROJECT; git checkout jenkins) ... $ python2.7 tools/perform-build.py --project PROJECT --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/PROJECT ... }}} or alternatively by using [https://projects.kde.org/projects/playground/sdk/macports-kde/repository/revisions/master/show/contrib/scripts/KDECI scripts from our MacPorts/KDE git repository] {{{ $ ./prepare.sh PROJECT ... $ ./build.sh PROJECT ... }}} where {{{prepare.sh}}} takes care of creating the build directory if it is not yet existing as well as checks out the branch 'jenkins'. This is only needed when building a project manually, otherwise it gets one by the jenkins slave. Alternatively one can combine both of these steps using: {{{ $ ./install.sh PROJECT }}} All frameworks of a specific tier can be build as a whole using {{{ $ # This builds all frameworks of tier 1 $ ./tier-install.sh 1 ... $ # This builds all frameworks of tier 2, but forces rebuilding every project regardless of $ # whether git pulled in changes or not (useful when e.g. qt5 or cmake was updated) $ ./tier-install.sh 2 rebuild ... }}} In order to be able to build KF5 frameworks of tier 3 it is - '''as a temporary workaround''' - necessary to copy kdoctools' files from its install directory to where framework builds can find them (i.e. below "/Library/Application Support"): {{{ $ sudo rm -rf /Library/Application\ Support/kf5/kdoctools/ ; sudo cp -Rp /opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdoctools/inst/Library/Application\ Support/kf5 /Library/Application\ Support }}} '''{It should be possible to avoid this by patching Qt5's QStandardPaths in an appropriate manner, which is atm still investigated...}''' KF5 builds have been successfully done so far for: {{{ Additional dependencies: - extra-cmake-modules - phonon Tier1 frameworks (COMPLETE): - attica - kapidox - karchive - kcodecs - kconfig - kcoreaddons - kdbusaddons - kdnssd - { kglobalaccel } - kguiaddons - ki18n - kidletime - kimageformats - kitemmodels - kitemviews - kplotting - kwidgetsaddons - { kwindowsystem } - solid - sonnet - threadweaver Tier2 frameworks (COMPLETE): - kauth - kcompletion - { kcrash } - kdoctools - kjobwidgets - { kpty } - kunitconversion Tier3 frameworks incl. phonon (ALMOST COMPLETE): [ATTENTION: (1) above workaround needed, (2) the build order matters here] - kconfigwidgets - kservice - kiconthemes - ktextwidgets - kxmlgui - kcmutils - kbookmarks - { kdesu } - kemoticons - knotifications - kwallet - kio - kactivities - kdeclarative - kparts - kdewebkit - kinit - kmediaplayer - knewstuff - knotifyconfig - ktexteditor - kdesignerplugin - kded - plasma-framework Tier 4: - frameworkintegration Porting aids: - kjs - kdelibs4support - kjsembed - khtml - kross - krunner Meta-project including tiers 1-4 + porting aids: - kf5umbrella }}} Curly brackets in the above lists mark frameworks marked on [http://api.kde.org/frameworks-api/frameworks5-apidocs/ KDE's official API page] as having problems on MacOSX, since plasma won't be running for now. Having the very compact [http://wiki.gentoo.org/wiki/Project:KDE/Frameworks#kdelibs Gentoo's Project:KDE/Frameworks wiki page] around turned out to be very helpful while building the above KDE frameworks manually. More info is found on [http://api.kde.org/frameworks-api/frameworks5-apidocs/ KDE's official API page] and the above lists have been aligned with it: nice dependency graphs for every framework can be found there by clicking the "Dependencies" link on each frameworks page, some more (overview) graphs are [http://agateau.com/2013/12/05/kf5-diagrams/ here]. '''As of July 5th 2014 all KF5 frameworks build successfully.''' {NOTE: Still using the above workaround for the CI system!} == Notes concerning some files installed by the frameworks == === Installs into "/Library/Application Support" === These folders should be copied into the MacPorts/KDE-CI system's "/Library/Application Support" directory (as it was done for as a temporary work-around for kdoctools above): {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/Library/Application Support/*" | egrep "/kf5$" ./frameworkintegration/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/frameworkintegration/inst/Library/Application Support/kf5 ./kauth/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kauth/inst/Library/Application Support/kf5 ./kconfigwidgets/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kconfigwidgets/inst/Library/Application Support/kf5 ./kdelibs4support/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdelibs4support/inst/Library/Application Support/kf5 ./kdesignerplugin/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdesignerplugin/inst/Library/Application Support/kf5 ./kdoctools/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdoctools/inst/Library/Application Support/kf5 ./khtml/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/khtml/inst/Library/Application Support/kf5 ./kio/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kio/inst/Library/Application Support/kf5 ./kjs/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjs/inst/Library/Application Support/kf5 ./knewstuff/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/knewstuff/inst/Library/Application Support/kf5 ./kwidgetsaddons/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kwidgetsaddons/inst/Library/Application Support/kf5 ./kxmlgui/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kxmlgui/inst/Library/Application Support/kf5 ./sonnet/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/sonnet/inst/Library/Application Support/kf5 }}} === Problematic installs into "/Library/Application Support" === The following folders do not reside below a kf5 sub-directory of "/Library/Application Support" which seems to indicate that there is a problem in the corresponding CMake files of {{{khtml}}} and {{{ktexteditor}}} frameworks: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/Library/Application Support/*" | grep -v "/kf5/" | egrep -v "/kf5$" ./khtml/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/khtml/inst/Library/Application Support/khtml ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5 ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/commands ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files/quickcoding ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files/quickcoding/cpp ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/indentation ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/libraries ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/libraries/emmet ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/syntax }}} === Installs into "/Applications/KF5" === These are all application packages which might have to be copied into {{{/Applications/KF5}}} on the KDE/CI system: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/frameworks/*/Applications/KF5/*.app" ./kactivities/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kactivities/inst/Applications/KF5/kactivitymanagerd.app ./kded/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kded/inst/Applications/KF5/kded5.app ./kdelibs4support/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdelibs4support/inst/Applications/KF5/kdebugdialog5.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/Applications/KF5/kdeinit5.app ./kjsembed/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjsembed/inst/Applications/KF5/kjscmd5.app ./kross/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kross/inst/Applications/KF5/kf5kross.app ./kwallet/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kwallet/inst/Applications/KF5/kwalletd5.app }}} === Installs NOT into "/Applications/KF5" === These are all application packages which MIGHT NOT have to be copied into {{{/Applications/KF5}}} on the KDE/CI system: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/frameworks/*.app" | grep -v "Applications/KF5" ./kauth/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kauth/inst/lib/libexec/kauth/kauth-policy-gen.app ./kconfig/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kconfig/inst/lib/libexec/kf5/kconfig_compiler_kf5.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/lib/libexec/kf5/start_kdeinit.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/lib/libexec/kf5/start_kdeinit_wrapper.app ./kio/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kio/inst/lib/libexec/kf5/kio_http_cache_cleaner.app ./kjsembed/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjsembed/inst/bin/kjsconsole.app }}} == kdesupport-svn - a prerequisite for further projects == In order to be able to build some KDE applications it is necessary to install [https://techbase.kde.org/Getting_Started/Build/KDE_Support#kdesupport_svn_module kdesupport-svn], which is not automatically handled by the prepare script. Therefore one needs to check out its sources using Subversion and install it like this: {{{ $ mkdir ~/WC/KDECI-builds/kdesupport-svn $ svn checkout svn://anonsvn.kde.org/home/kde/trunk/kdesupport ~/WC/KDECI-builds/kdesupport-svn $ ./build.sh kdesupport-svn }}} == KF5 software beyond the basic frameworks == Since the basic set of frameworks is now ready for use on OSX one can go ahead and start building those. {{{ Done: - kde-cli-tools - khelpcenter (-) - kate (-) - libkomparediff2 - kfilemetadata - kdevplatform (1) - kinfocenter (+) Planned: - develop (2) - kdepimlibs (3) - kmymoney (4) KDE edu: - analitza (5) - kalgebra - kgeography (6) - step (7) Unneeded on OSX (tried just for the fun of it): - libkscreen (a) - kde-runtime (a,b) - ktp-common-internals (c) - libksysguard (d) - khotkeys (e) - kmenuedit (e) - milou (f) - powerdevil (f) - baloo (g) - xapian }}} (-) crashes on startup ([https://bugs.kde.org/show_bug.cgi?id=337140 b.k.o. issue 337140]) and needs DYLD_LIBRARY_PATH set for phonon[[BR]] (+) crashes on startup[[BR]] (1) kdevplatform needs [http://mail.kde.org/pipermail/kdevelop/2014-July/018384.html patch][[BR]] (2) kdevelop [http://mail.kde.org/pipermail/kdevelop/2014-July/018388.html fails][[BR]] (3) needs akonadi[[BR]] (4) kmymoney misses [http://mail.kde.org/pipermail/kmymoney-devel/2014-July/012773.html some frameworks][[BR]] (5) configure problem (requires kde4-config for unknown reason) (6) docbook error ("Entity 'language' not defined") (7) MacPorts package eigen not found by FW step (a) FWs require Qt5X11Extras, although it is switched off in global configuration using -DCMAKE_DISABLE_FIND_PACKAGE_X11=ON[[BR]] (b) kde-runtime is not needed, since its functionality is currently getting ported into other KF5 FWs step by step[[BR]] (c) ktp-common-internals requires TelepathyQt5[[BR]] (d) libksysguard is [http://mail.kde.org/pipermail/kde-frameworks-devel/2014-July/017312.html broken] (see also [http://stackoverflow.com/a/19637199/2858390 this])[[BR]] (e) need libksysguard[[BR]] (f) needs baloo[[BR]] (g) needs [http://xapian.org xapian][[BR]]