[[PageOutline]] = Current status of setting up the CI system = This page shall describe what has to be done to get a KDE/CI system up and running on OSX. Partially this is achieved by using a few ports from MacPorts. == Start with a fresh Mavericks/MacPorts install == Currently using the official MacPorts version 2.3.1 is recommended. To be on the safe side make sure your disk has about 60G space in total. However, currently a Mavericks 10.9.2 KDE/CI system on an i7-iMac uses something like this: {{{ $ df -h / Filesystem Size Used Avail Use% Mounted on /dev/disk0s2 64G 36G 29G 56% / }}} that is after all frameworks have been built and installed as shown further down. == Getting a MacPorts-compatible bash environment == Install ports for MacPorts' bash {{{ $ sudo port install bash bash-completion }}} and set up the [https://projects.kde.org/projects/playground/sdk/macports-kde/repository/revisions/master/show/contrib/user-setup bash environment for MacPorts use] regarding search paths (PATH, MANPATH) == Apple developer tools == Install * Xcode from AppStore (and agree actively to the developer license agreement) * developer command line tools {{{ $ xcode-select --install }}} == Tools already present on OSX == Some tools are already installed on OSX due to Xcode (like git, svn, ssh, rsync, make, clang etc.) they don't need to be installed via MacPorts for now. Should a more up-to-date version of any one of these be necessary for the CI system it could be added later. == Some more software supplied by MacPorts == Install ports needed for the CI scripts as well as KF5: {{{ $ sudo port install gnutar py27-lxml bazaar mercurial wget automake autoconf cppcheck shared-mime-info docbook-xml docbook-xsl giflib boost }}} == Basic setup of CI system == Create some folders, clone and update CI system (assuming a user "kdeci" with its home directory "/Users/kdeci"): {{{ $ cd $ mkdir -p WC/KDECI-build; $ cd WC $ git clone git://anongit.kde.org/websites/build-kde-org $ cd build-kde-org $ git checkout production $ ./update-setup.sh # Ignore 404 occurring due to error cloning ECMA262 by hg }}} The update shell script created a folder {{{~/scripts}}} in which subsequent commands will have to be executed. == 1st step: Qt5 build == === Prepare environment for Qt5 build === Install additional ports specific for Qt5 as well as KF5, set correct path to KDE-install directory in config/build/darwin-mavericks.cfg: {{{ $ sudo port install zlib openssl dbus jpeg tiff libmng libpng mysql55 pkgconfig sqlite2 $ sudo port uninstall cmake # Remove MacPorts' cmake (which came into the system as build-dependency for mysql55) $ cd ~/scripts $ cat config/build/darwin-mavericks.cfg [DEFAULT] scriptsLocation=/Users/kdeci/scripts opSys=darwin architecture=mavericks compiler=clang [General] installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ $ python2.7 tools/prepare-environment.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/qt5 ... $ }}} === Set up build environment for Qt5 === Make sure all libs can be found by adding search paths and avoid inclusion of MacPorts' glib2: {{{ $ cd ~/scripts; cat config/build/qt5/darwin-mavericks.cfg [Build] configureCommand=%(configureExecutable)s -release -system-zlib -system-libpng -system-libjpeg -system-sqlite -dbus -plugin-sql-mysql -nomake examples -confirm-license -opensource -prefix {instPrefix} -no-framework -v -I/opt/local/include -L/opt/local/lib -no-glib }}} Consider here the use of options "{{{-debug -separate-debug-info}}}" instead of "{{{-release}}}". Include MySQL's binary path into PATH e.g. in {{{.macports/profile}}} (assuming /opt/local as MacPort's prefix): {{{ export PATH=/opt/local/lib/mysql55/bin:$PATH }}} Set up SSH for rsync access to build host, then start building Qt5: {{{ $ cat ~/.ssh/config Host build.kde.org Port 2022 $ ssh-keygen -C YOUR@EMAIL.ADDRESS # <-- send your public SSH key to Ben Cooksley in order to get access to the build server $ python2.7 tools/perform-build.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/qt5 ... }}} == Installation of cmake == Cmake needs a manual checkout of its sources: {{{ $ python2.7 tools/prepare-environment.py --project cmake --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/cmake ... $ git clone git://cmake.org/cmake.git ~/WC/KDECI-build/cmake $ python2.7 tools/perform-build.py --project cmake --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/cmake ... }}} == Keeping build environment up-to-date == For KF5 one has to modify the configuration settings by commenting out the kauth dependency on polkit-qt-1, since that dependency is only needed on Linux: {{{ $ cd ~/scripts $ ./update-setup.sh $ vi config/base/kf5-qt5 $ grep kauth config/base/kf5-qt5 #frameworks/kauth: kdesupport/polkit-qt-1 $ }}} Problems appeared for some frameworks since kconfig_compiler_kf5.app doesn't yet get installed on OSX (the build scripts error out with "/Applications/KDE/kconfig_compiler_kf5.app/Contents/MacOS/kconfig_compiler_kf5 as location is wrong"). This can be fixed for now introducing a temporary configuration file for kconfig: {{{ $ cat ~/scripts/config/build/kconfig/darwin-mavericks.cfg [DEFAULT] configureExtraArgs=-DCMAKE_INSTALL_BUNDLEDIR="lib/libexec/kf5" }}} which makes sure that the application bundle gets installed below PREFIX/lib/libexec/. '''{It seems like that this specific issue has been identified as a kconfig bug and got resolved in the meantime in revision 70193760afc52ebc1ab99c9e467b9df470d9e39a and the workaround is therefore not needed anymore!}''' In order to always have the latest KDE/CI running it makes sense to temporally stash away the changes made locally and upgrade via git {{{ $ # Check what has changed locally: $ git diff diff --git a/config/base/kf5-qt5 b/config/base/kf5-qt5 index 16abdf6..fff9647 100644 --- a/config/base/kf5-qt5 +++ b/config/base/kf5-qt5 @@ -18,7 +18,7 @@ general/vc: -qt5 general/shared-desktop-ontologies: -qt5 # KDE Frameworks -frameworks/kauth: kdesupport/polkit-qt-1 +#frameworks/kauth: kdesupport/polkit-qt-1 kde/*: general/kdesupport-svn kde/workspace/plasma-workspace: general/libdbusmenu-qt diff --git a/config/build/darwin-mavericks.cfg b/config/build/darwin-mavericks.cfg index cb62c49..0648464 100644 --- a/config/build/darwin-mavericks.cfg +++ b/config/build/darwin-mavericks.cfg @@ -1,7 +1,9 @@ [DEFAULT] +scriptsLocation=/Users/kdeci/scripts opSys=darwin architecture=mavericks compiler=clang +configurePlatformArgs=-DCMAKE_INSTALL_BUNDLEDIR="Applications/KF5" -DDATA_INSTALL_DIR="Library/Application Support" -DCMAKE_DISABLE_FIND_PACKAGE_X11=ON [General] -installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ \ No newline at end of file +installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ diff --git a/tools/environment-generator.py b/tools/environment-generator.py index 37dcbae..e46a13c 100644 --- a/tools/environment-generator.py +++ b/tools/environment-generator.py @@ -31,7 +31,7 @@ environment = manager.generate_environment(True) neededVariables = [ 'CMAKE_PREFIX_PATH', 'KDEDIRS', 'PATH', 'LD_LIBRARY_PATH', 'PKG_CONFIG_PATH', 'PYTHONPATH', 'PERL5LIB', 'QT_PLUGIN_PATH', 'QML_IMPORT_PATH', 'QML2_IMPORT_PATH', 'XDG_DATA_DIRS', - 'XDG_CONFIG_DIRS', 'QMAKEFEATURES', 'XDG_CURRENT_DESKTOP' + 'XDG_CONFIG_DIRS', 'QMAKEFEATURES', 'XDG_CURRENT_DESKTOP', 'DYLD_LIBRARY_PATH' ] # Generate the shell format environment file, suitable for sourcing $ $ git status On branch production Your branch is up-to-date with 'origin/production'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: config/base/kf5-qt5 modified: config/build/darwin-mavericks.cfg Untracked files: (use "git add ..." to include in what will be committed) build.sh dependencies/ kapidox/ kde_projects.xml poppler-test-data/ prepare.sh no changes added to commit (use "git add" and/or "git commit -a") $ $ # Updating CI scripts $ cd ~/scripts; git stash; git pull; git stash pop }}} == Installation of other projects and KDE frameworks == The various frameworks have to be handled by replacing "PROJECT" with the framework's project name: {{{ $ python2.7 tools/prepare-environment.py --project PROJECT --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/PROJECT ... $ (cd ~/WC/KDECI-build/PROJECT; git checkout jenkins) ... $ python2.7 tools/perform-build.py --project PROJECT --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/PROJECT ... }}} or alternatively by using [https://projects.kde.org/projects/playground/sdk/macports-kde/repository/revisions/master/show/contrib/scripts/KDECI scripts from our MacPorts/KDE git repository] {{{ $ ./prepare.sh PROJECT ... $ ./build.sh PROJECT ... }}} where {{{prepare.sh}}} takes care of creating the build directory if it is not yet existing as well as checks out the branch 'jenkins'. This is only needed when building a project manually, otherwise it gets one by the jenkins slave. Alternatively one can combine both of these steps using: {{{ $ ./install.sh PROJECT }}} All frameworks of a specific tier can be build as a whole using {{{ $ # This builds all frameworks of tier 1 $ ./tier-install.sh 1 ... $ # This builds all frameworks of tier 2, but forces rebuilding every project regardless of $ # whether git pulled in changes or not (useful when e.g. qt5 or cmake was updated) $ ./tier-install.sh 2 rebuild ... }}} In order to be able to build KF5 frameworks of tier 3 it is - '''as a temporary workaround''' - necessary to copy kdoctools' files from its install directory to where framework builds can find them (i.e. below "/Library/Application Support"): {{{ $ sudo rm -rf /Library/Application\ Support/kf5/kdoctools/ ; sudo cp -Rp /opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdoctools/inst/Library/Application\ Support/kf5 /Library/Application\ Support }}} '''{It should be possible to avoid this by patching Qt5's QStandardPaths in an appropriate manner, which is atm still investigated...}''' KF5 builds have been successfully done so far for: {{{ Additional dependencies: - extra-cmake-modules - phonon Tier1 frameworks (COMPLETE): - attica - kapidox - karchive - kcodecs - kconfig - kcoreaddons - kdbusaddons - kdnssd - { kglobalaccel } - kguiaddons - ki18n - kidletime - kimageformats - kitemmodels - kitemviews - kplotting - kwidgetsaddons - { kwindowsystem } - solid - sonnet - threadweaver Tier2 frameworks (COMPLETE): - kauth - kcompletion - { kcrash } - kdoctools - kjobwidgets - { kpty } - kunitconversion Tier3 frameworks incl. phonon (ALMOST COMPLETE): [ATTENTION: (1) above workaround needed, (2) the build order matters here] - kconfigwidgets - kservice - kiconthemes - ktextwidgets - kxmlgui - kcmutils - kbookmarks - { kdesu } - kemoticons - knotifications - kwallet - kio - kactivities - kdeclarative - kparts - kdewebkit - kinit - kmediaplayer - knewstuff - knotifyconfig - ktexteditor - kdesignerplugin - kded - plasma-framework Tier 4: - frameworkintegration Porting aids: - kjs - kdelibs4support - kjsembed - khtml - kross - krunner Meta-project including tiers 1-4 + porting aids: - kf5umbrella }}} Curly brackets in the above lists mark frameworks marked on [http://api.kde.org/frameworks-api/frameworks5-apidocs/ KDE's official API page] as having problems on MacOSX, since plasma won't be running for now. Having the very compact [http://wiki.gentoo.org/wiki/Project:KDE/Frameworks#kdelibs Gentoo's Project:KDE/Frameworks wiki page] around turned out to be very helpful while building the above KDE frameworks manually. More info is found on [http://api.kde.org/frameworks-api/frameworks5-apidocs/ KDE's official API page] and the above lists have been aligned with it: nice dependency graphs for every framework can be found there by clicking the "Dependencies" link on each frameworks page, some more (overview) graphs are [http://agateau.com/2013/12/05/kf5-diagrams/ here]. '''As of July 5th 2014 all KF5 frameworks build successfully.''' {NOTE: Still using the above workaround for the CI system!} == Notes concerning some files installed by the frameworks == === Installs into "/Library/Application Support" === These folders should be copied into the MacPorts/KDE-CI system's "/Library/Application Support" directory (as it was done for as a temporary work-around for kdoctools above): {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/Library/Application Support/*" | egrep "/kf5$" ./frameworkintegration/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/frameworkintegration/inst/Library/Application Support/kf5 ./kauth/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kauth/inst/Library/Application Support/kf5 ./kconfigwidgets/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kconfigwidgets/inst/Library/Application Support/kf5 ./kdelibs4support/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdelibs4support/inst/Library/Application Support/kf5 ./kdesignerplugin/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdesignerplugin/inst/Library/Application Support/kf5 ./kdoctools/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdoctools/inst/Library/Application Support/kf5 ./khtml/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/khtml/inst/Library/Application Support/kf5 ./kio/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kio/inst/Library/Application Support/kf5 ./kjs/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjs/inst/Library/Application Support/kf5 ./knewstuff/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/knewstuff/inst/Library/Application Support/kf5 ./kwidgetsaddons/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kwidgetsaddons/inst/Library/Application Support/kf5 ./kxmlgui/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kxmlgui/inst/Library/Application Support/kf5 ./sonnet/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/sonnet/inst/Library/Application Support/kf5 }}} === Problematic installs into "/Library/Application Support" === The following folders do not reside below a kf5 sub-directory of "/Library/Application Support" which seems to indicate that there is a problem in the corresponding CMake files of {{{khtml}}} and {{{ktexteditor}}} frameworks: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/Library/Application Support/*" | grep -v "/kf5/" | egrep -v "/kf5$" ./khtml/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/khtml/inst/Library/Application Support/khtml ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5 ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/commands ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files/quickcoding ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files/quickcoding/cpp ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/indentation ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/libraries ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/libraries/emmet ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/syntax }}} === Installs into "/Applications/KF5" === These are all application packages which might have to be copied into {{{/Applications/KF5}}} on the KDE/CI system: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/frameworks/*/Applications/KF5/*.app" ./kactivities/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kactivities/inst/Applications/KF5/kactivitymanagerd.app ./kded/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kded/inst/Applications/KF5/kded5.app ./kdelibs4support/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdelibs4support/inst/Applications/KF5/kdebugdialog5.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/Applications/KF5/kdeinit5.app ./kjsembed/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjsembed/inst/Applications/KF5/kjscmd5.app ./kross/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kross/inst/Applications/KF5/kf5kross.app ./kwallet/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kwallet/inst/Applications/KF5/kwalletd5.app }}} === Installs NOT into "/Applications/KF5" === These are all application packages which MIGHT NOT have to be copied into {{{/Applications/KF5}}} on the KDE/CI system: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/frameworks/*.app" | grep -v "Applications/KF5" ./kauth/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kauth/inst/lib/libexec/kauth/kauth-policy-gen.app ./kconfig/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kconfig/inst/lib/libexec/kf5/kconfig_compiler_kf5.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/lib/libexec/kf5/start_kdeinit.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/lib/libexec/kf5/start_kdeinit_wrapper.app ./kio/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kio/inst/lib/libexec/kf5/kio_http_cache_cleaner.app ./kjsembed/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjsembed/inst/bin/kjsconsole.app }}} == kdesupport-svn - a prerequisite for further projects == In order to be able to build some KDE applications it is necessary to install [https://techbase.kde.org/Getting_Started/Build/KDE_Support#kdesupport_svn_module kdesupport-svn], which is not automatically handled by the prepare script. Therefore one needs to check out its sources using Subversion and install it like this: {{{ $ mkdir ~/WC/KDECI-builds/kdesupport-svn $ svn checkout svn://anonsvn.kde.org/home/kde/trunk/kdesupport ~/WC/KDECI-builds/kdesupport-svn $ ./build.sh kdesupport-svn }}} == KF5 software beyond the basic frameworks == Since the basic set of frameworks is now ready for use on OSX one can go ahead and start building those. {{{ Done: - kde-cli-tools - khelpcenter - kate - libkomparediff2 - kfilemetadata - kdevplatform (a) - kinfocenter Planned: - develop (b) - kdepimlibs (c) - kmymoney (d) Just for the fun of it: - libkscreen (e) - kde-runtime (e) - ktp-common-internals (f) - libksysguard (g) - khotkeys (h) - kmenuedit (h) - milou (i) - powerdevil (i) - baloo (j) - xapian }}} (a) kdevplatform needs [http://mail.kde.org/pipermail/kdevelop/2014-July/018384.html patch][[BR]] (b) kdevelop [http://mail.kde.org/pipermail/kdevelop/2014-July/018388.html fails][[BR]] (c) needs akonadi[[BR]] (d) kmymoney misses [http://mail.kde.org/pipermail/kmymoney-devel/2014-July/012773.html some frameworks][[BR]] (e) frameworks require Qt5X11Extras, although it is switched off in global configuration using -DCMAKE_DISABLE_FIND_PACKAGE_X11=ON[[BR]] (f) ktp-common-internals requires TelepathyQt5[[BR]] (g) libksysguard is [http://mail.kde.org/pipermail/kde-frameworks-devel/2014-July/017312.html broken][[BR]] (h) need libksysguard[[BR]] (i) needs baloo[[BR]] (j) needs [http://xapian.org xapian][[BR]]