1 [07:00] <@dholbach> GOOD MORNING EVERYBODY!
   2 [07:00] <@dholbach> who do we have here for the Packaging Training session? :)
   3 [07:01] <rww> o/
   4 [07:01] <genii> It's morning?
   5 [07:01] <rww> genii: in crazy UTC people time, yeah. 11pm here :(
   6 [07:01] <@dholbach> genii: it is over here - 08:01 right now :-)
   7 [07:02] <Rail> moin moin
   8 [07:02] <@dholbach> so... do we have any questions already before we kick off the session?
   9 [07:03] <Crusher> nope
  10 [07:03] <@dholbach> alright... :-)
  11 [07:03]  * jost86 gets Coffee...
  12 [07:03] <@dholbach> https://wiki.ubuntu.com/MOTU/GettingStarted is the one page you absolutely need to bookmark :)
  13 [07:04] <@dholbach> it contains links to all important documentation
  14 [07:04] <@dholbach> like the Packaging Guide, like the Developer Videos, Ubuntu Developer process documentation, simple bugs to start working on etc
  15 [07:05] <@dholbach> ok
  16 [07:05] <@dholbach> which version of Ubuntu are you all running?
  17 [07:05] <Crusher> jaunty
  18 [07:05] <djails> Hardy
  19 [07:05] <rww> Jaunty :)
  20 [07:05] <jost86> jaunty
  21 [07:06] <genii> 8.04.2
  22 [07:06] <@dholbach> alright :)
  23 [07:06] <zaidka> hardy
  24 [07:06] <slangasek> breezy
  25 [07:06] <zaidka> kidding.. jaunty
  26 [07:06] <maxb> jaunty with a karmic install available for dual-booting :-)
  27 [07:06] <sebsebseb> upgraded 8.10,  to 9.04 alpha6, to beta, to  rc.  install that has some issues from alpha6, so will eventualy clean install
  28 [07:06] <zaidka> ubuntu cambridge
  29 [07:06] <@dholbach> if you want to work on Ubuntu, you absolutely need to have the latest development release
  30 [07:07] <@dholbach> https://wiki.ubuntu.com/UbuntuDevelopment/UsingDevelopmentReleases explains how to do that in a sane way :)
  31 [07:07] <@dholbach> I personally had chroots (explained on there as well) for a long time, but now I almost exclusively use kvm (used virtualbox in the past)
  32 [07:08] <nixternal> woowoo
  33 [07:08] <@dholbach> you absolutely need to be able to test packages in the ubuntu version you're about to upload them for :-)
  34 [07:08] <nixternal> teach me ol' great one!
  35 [07:09] <@dholbach> nixternal: come on... you've been around long enough already :)
  36 [07:09]  * genii feeds nixternal some pie to quieten him down
  37 [07:09] <nixternal> mmmmm
  38 [07:09] <@dholbach> for now, let's just keep using the version we're all comfortable with, but please make a note to set up a VM at some stage :-)
  39 [07:09] <@dholbach> please run:
  40 [07:09] <@dholbach>   sudo apt-get install ubuntu-dev-tools devscripts build-essential gnupg
  41 [07:10] <@dholbach> it will install a bunch of packages for you we're going to need for the session
  42 [07:10] <nixternal> isn't gnupg installed by default already?
  43 [07:10] <@dholbach> maybe nixternal can exaplain what they're there for
  44 [07:10] <@dholbach> nixternal: just making sure :)
  45 [07:10] <nixternal> k, confused me for a sec
  46 [07:10] <raylu> if we're packaging for the current release, we won't need the latest dev release, right?
  47 [07:10] <zaidka> what! none of them is installed by default
  48 [07:10] <genii> I have to beg out, I'll be /away but checking the backscroll later
  49 [07:10] <@dholbach> raylu: for now we'll just play around with the tools, set things up and so on
  50 [07:11] <nixternal> raylu: correct, though you never know when you will want to start developing for the next release
  51 [07:11] <@dholbach> raylu: if you want to upload packages for karmic, you need to have them built and tested on karmic :)
  52 [07:11] <@dholbach> raylu: I hope that makes sense
  53 [07:11]  * nixternal notes to fix it in the dev cycle and backport/sru as needed to the current stable release
  54 [07:11]  * ara remembers that if there is someone that prefer to make questions in Spanish, they can be posted at #ubuntu-classroom-chat-es
  55 [07:11] <nixternal> thanks ara!
  56 [07:12] <raylu> dholbach: right; just that you said we "absolutely" need the latest dev release
  57 [07:12]  * nixternal goes back to watching and hushes up
  58 [07:12] <@dholbach> raylu: right, "absolutely" for testing :-)
  59 [07:12] <@dholbach> and "in some form" :)
  60 [07:12] <@dholbach> ok
  61 [07:12] <@dholbach> so this is what the packages are for:
  62 [07:12] <rww> "Please select the mail server configuration type that best meets your needs." =/
  63 [07:13] <@dholbach> rww: you can purge mailx and the mailserver afterwards - sorry
  64 [07:13] <@dholbach> or use --no-install-recommends
  65 [07:13] <@dholbach> (in the apt-get call)
  66 [07:13] <nixternal> rww: just do the top selection, no configuration for now
  67 [07:13] <@dholbach>  - gnupg (installed by default as nixternal said) can sign and encrypt files (we use it to sign source packages)
  68 [07:14] <@dholbach>  - build-essential: will install the bare minimum to be able to compile (very simple packages): the compiler, make, etc.
  69 [07:15] <@dholbach>  - devscripts: a bunch of tools that make Debian packaging much easier (especially repetitive tasks)
  70 [07:15] <@dholbach>  - ubuntu-dev-tools: much like devscripts, but more closely tied to Ubuntu
  71 [07:15] <@dholbach> any questions so far?
  72 [07:15] <jost86> nope
  73 [07:16] <teknico> I got here late
  74 [07:16] <slangasek> using --no-install-recommends (or setting it in /etc/apt/apt.conf.d/) is highly advisable for a development chroot, not just because it saves download/configuration time, but because that's a more accurate model of how the buildds work
  75 [07:16] <teknico> are there logs anywhere?
  76 [07:16] <@dholbach> teknico: sudo apt-get install --no-install-recommends ubuntu-dev-tools devscripts build-essential gnupg    (is where we are right now)
  77 [07:17] <@dholbach> plus a bunch of links that will be available on https://wiki.ubuntu.com/Packaging/Training later on
  78 [07:17] <teknico> thx
  79 [07:17] <@dholbach> ok, please edit   ~/.pbuilderrc    in your favourite editor and add the following
  80 [07:17] <@dholbach> COMPONENTS="main universe multiverse restricted"
  81 [07:17] <@dholbach> and then save the file
  82 [07:17] <rww> teknico: http://paste.ubuntu.com/161169/
  83 [07:18] <@dholbach> we often use a tool called pbuilder which will build source packages in a minimal environment for us (sets up that environment, installs the build-depends packages, builds the package, etc in an automated fashion)
  84 [07:19] <@dholbach> for now just run
  85 [07:19] <@dholbach>    pbuilder-dist create <your Ubuntu version>         (hardy, jaunty, karmic, whatever)
  86 [07:19] <@dholbach> it will set up the initial pbuilder
  87 [07:19] <@dholbach> the nice thing about pbuilder-dist is, that you can set up different build environments for karmic, jaunty, intrepid, hardy, etc etc :)
  88 [07:20] <@dholbach> (also Debian versions)
  89 [07:20] <@dholbach> so that's pretty neatr
  90 [07:20] <@dholbach> neat
  91 [07:20] <Crusher> dholbach: if you already have a pbuilder environment setup, will this overwrite it?
  92 [07:20] <@dholbach> https://wiki.ubuntu.com/PbuilderHowto has more info about that tool
  93 [07:20] <@dholbach> Crusher: did you use pbuilder-dist?
  94 [07:20] <rww> dholbach: are "create" and "<your ubuntu version>" switched around? I'm getting an error
  95 [07:20] <Crusher> dholbach: no
  96 [07:20] <@dholbach> rww: they are :-)
  97 [07:20] <@dholbach>    pbuilder-dist <your Ubuntu version> create        (hardy, jaunty, karmic, whatever)
  98 [07:20] <@dholbach> thanks a lot rww
  99 [07:21]  * dholbach didn't sleep very well last night *yawn*
 100 [07:21] <@dholbach> Crusher: in that case it shouldn't, but you can safely skip the step as well
 101 [07:21] <@dholbach> ok... while that's running, let's set up a GPG key
 102 [07:21] <@dholbach> who of you does not have a GPG key set up yet?
 103 [07:22] <jost86> i might have one... but I dont remember it
 104 [07:22] <@dholbach> awesome, looks like we can skip a step :)
 105 [07:22] <jost86> well go ahead
 106 [07:22] <@dholbach> jost86: what does     ls ~/.gnupg/pubring.gpg     say?
 107 [07:22] <@dholbach> do you have that file?
 108 [07:23] <jost86> yep, present
 109 [07:23] <@dholbach> super
 110 [07:23] <@dholbach> basically I would have run you through the steps of:
 111 [07:23] <@dholbach>   gpg --gen-key
 112 [07:24] <@dholbach>   gpg --send-keys <key>
 113 [07:24] <@dholbach> and putting it up on Launchpad
 114 [07:24] <raylu> i'd "gpg -K" to make sure you actually have a key
 115 [07:24] <@dholbach> raylu: good one - great
 116 [07:24] <jost86> k, i think i'll figure that out :) thx
 117 [07:25] <@dholbach> if you didn't put up your GPG key on Launchpad, you can do it here:   https://launchpad.net/people/+me/+editpgpkeys
 118 [07:26] <@dholbach> in the beginning you won't need the GPG key very often, as you're not directly allowed to upload packages to Ubuntu yet
 119 [07:26] <@dholbach> but I'll get back to the "upload packages part" in a bit
 120 [07:26] <@dholbach> if you use bash (default shell), please add something like this to your ~/.bashrc
 121 [07:26] <@dholbach> export DEBFULLNAME='Daniel Holbach'
 122 [07:26] <@dholbach> export DEBEMAIL='daniel.holbach@ubuntu.com'
 123 [07:27] <@dholbach> these are variables that are used by the tools in devscripts and it'll save you typing your name over and over again :)
 124 [07:27] <@dholbach> (editing changelog entries is one of the very common use-cases)
 125 [07:28] <@dholbach> another thing you need to make sure when you're working on Ubuntu packages is that you have source package repositories enabled in apt
 126 [07:28] <@dholbach> so you need something like this in your /etc/apt/sources.list
 127 [07:28] <@dholbach> deb-src http://archive.ubuntu.com/ubuntu/ jaunty restricted main multiverse universe
 128 [07:28] <@dholbach> (of course replace jaunty with karmic or hardy or whatever you're using)
 129 [07:29] <@dholbach> you can also use the System -> Administration -> Software Sources tool
 130 [07:29] <@dholbach> (or whatever it's called in English, sorry :-))
 131 [07:29] <@dholbach> once you've configured that, you can simply run
 132 [07:30] <@dholbach>    sudo apt-get update
 133 [07:30] <@dholbach>    apt-get source hello
 134 [07:30] <@dholbach> and it will download you the current source code for that package from the archive
 135 [07:30] <@dholbach> any questions up until now?
 136 [07:31] <@dholbach> so let's run
 137 [07:31] <@dholbach>   apt-get source hello
 138 [07:31] <@dholbach> it's a very very boring package, but we'll download it anyway now
 139 [07:31] <@dholbach> apt is going to download the following files:
 140 [07:31] <@dholbach>  hello_2.2-3.diff.gz
 141 [07:32] <@dholbach>  hello_2.2-3.dsc
 142 [07:32] <@dholbach>  hello_2.2.orig.tar.gz
 143 [07:32] <@dholbach> (might be other versions on your system)
 144 [07:32] <@dholbach> let's go through them one by one
 145 [07:33] <@dholbach> the .orig.tar.gz file is the original pristine untouched tarball that was downloaded from the website of the software authors
 146 [07:33] <@dholbach> in this case it's from something like ftp.gnu.org
 147 [07:33] <@dholbach> it's important that we don't change it (only very rare cases where it's justified), but merely rename it to something like
 148 [07:33] <@dholbach>    <software project>_<version>.orig.tar.gz
 149 [07:34] <@dholbach> the .diff.gz file contains the compressed set of changes we need to apply to make the package build the Ubuntu or Debian way
 150 [07:34] <@dholbach> that's "the packaging"
 151 [07:35] <@dholbach> basically we add a directory called  ./debian/  to the source which further describes the package, the history of the packaging, the copyright and so on
 152 [07:36] <@dholbach> when I said "make it build the Ubuntu/Debian way", I meant that we wrap a build process around the existing build system that the upstream package offers
 153 [07:36] <@dholbach> in the standard case of a C+autotools project that would be something like   ./configure; make; sudo make install
 154 [07:36] <@dholbach> in a python+setuptools case that would be    python setup.py install   (or something like that)
 155 [07:37] <@dholbach> the reason for that is: we want just one way to build packages :-)
 156 [07:37] <@dholbach> the .dsc file contains just some bits of metadata, like the md5sum of the files, etc.
 157 [07:37] <@dholbach> are there any questions up until now?
 158 [07:38] <Crusher> dholbach: under what circumstances do we make native packages?
 159 [07:38] <@dholbach> Crusher: good question
 160 [07:38] <rww> dholbach: I have a directory hello-2.2 with a bunch of files in it too, that you didn't mention...
 161 [07:38] <@dholbach> rww: will get to that in a sec
 162 [07:38] <@dholbach> it seems that Crusher has read a few chapters of the packaging guide already :-)
 163 [07:38] <rww> dholbach: oh, okay. Just checking I didn't do something wrong :)
 164 [07:39] <Crusher> ;)
 165 [07:39] <@dholbach> rww: no, you're doing great :-)
 166 [07:39] <raylu> after you answer that... what if their source relies on a directory called "debian" already?
 167 [07:39] <@dholbach> so in the case of a native package, you just have a    <project>_<version>.tar.gz    and a    <project>_<version>.dsc     file
 168 [07:40] <@dholbach> which means all Packaging changes are in the .tar.gz file too
 169 [07:40] <@dholbach> so you can't easily tell apart which change was done by the packager and which came from upstream
 170 [07:41] <@dholbach> also it means that for every small packaging change (update the package description for example) you need to upload the whole tarball (might be several MB)
 171 [07:42] <@dholbach> in the other case you just upload the .diff.gz and the .dsc (ie: 1.2-1: upload the .tar.gz, 1.2-2: just upload the .diff.gz)
 172 [07:42] <@dholbach> ok, so why do we use that option?
 173 [07:43] <@dholbach> in some cases we are upstream of the package (and don't do regular .tar.gz releases) and keep everything in svn or bzr or git or whatever
 174 [07:43] <@dholbach> in that case, especially if it's a small package, we just dump everything in that .tar.gz file and make it a native package
 175 [07:44] <@dholbach> Crusher: did that make sense?
 176 [07:44] <Crusher> dholbach: yes
 177 [07:44] <@dholbach> raylu: some upstream maintainers decide to ship their own Debian packaging, so there's a debian/ directory already
 178 [07:45] <@dholbach> that's one of the rare cases where we repackage the tarball and rip out the existing packaging
 179 [07:45] <@dholbach> it makes reading the .diff.gz much easier
 180 [07:45] <nixternal> what about asking upstream to release a prestine tarball without a debian directory?
 181 [07:45] <@dholbach> nixternal: very good point, it makes a lot of sense to do that
 182 [07:45] <@dholbach> but in the meantime that's probably what you do
 183 [07:46] <@dholbach> alright... cracking on and getting to rww's question
 184 [07:46] <@dholbach> what about the hello-2.2 directory
 185 [07:46] <@dholbach> "apt-get source" (actually it's dpkg-source -x) is a clever tool - what it did for us was
 186 [07:46] <@dholbach>  - extract the tarball
 187 [07:47] <@dholbach>  - use the .diff.gz and patch the code with our packaging changes
 188 [07:47] <@dholbach> (and check the md5sums along the way)
 189 [07:47] <@dholbach> so if you check out the hello-2.2 directory, you'll see that there's a debian/ directory already
 190 [07:47] <@dholbach> now let's test-build the package
 191 [07:47] <@dholbach> please run
 192 [07:47] <@dholbach>    pbuilder-dist <your release> build hello_*.dsc
 193 [07:48] <@dholbach> this might take some time
 194 [07:48] <@dholbach> and I hope you all get addicted of watching build messages fly by and get involved with Ubuntu development :-)
 195 [07:49] <raylu> if i wanted that, i'd be using gentoo
 196 [07:49] <@dholbach> haha :)
 197 [07:49] <@dholbach>   ~/pbuilder/*_result/   should now contain the resulting .deb file
 198 [07:50] <@dholbach> raylu: right but in our case, you do one fix and 10 million other users benefit from it
 199 [07:51] <@dholbach> if you check out https://wiki.ubuntu.com/PackagingGuide/Recipes you'll find a bunch of examples of what you can do with the packaging tools and perform simpler tasks like "upgrade package to a new upstream", "make a small change", etc.
 200 [07:51] <@dholbach> once you created a small patch, you can use the Sponsorship Process to get it included in Ubuntu
 201 [07:51] <@dholbach> https://wiki.ubuntu.com/SponsorshipProcess has all the gory details
 202 [07:51] <@dholbach> so what's that for?
 203 [07:52] <@dholbach> you're only allowed to upload Ubuntu source packages directly when you're part of the development team
 204 [07:52] <@dholbach> until then you need to get your patches reviewed and then uploaded by somebody else
 205 [07:53] <@dholbach> so what you do is: attach the patch to a bug report on Launchpad (or link to the source package you uploaded somewhere)
 206 [07:53] <imbrandon> its not as scary as it sounds
 207 [07:53] <@dholbach> then subscribe the sponsors team
 208 [07:53] <@dholbach> and they'll review the patch, give you feedback, etc.
 209 [07:53] <@dholbach> then apply the patch to the source package, sign it with their GPG key and upload it for you
 210 [07:53] <@dholbach> imbrandon: exactly
 211 [07:53] <@dholbach> the sponsors team is a really friendly bunch and you'll learn a lot that way
 212 [07:54] <@dholbach> once you've been through that process a couple of times, people will tell you that you should join the team, and then it's time to send in a short "application"
 213 [07:54] <@dholbach> https://wiki.ubuntu.com/UbuntuDevelopers has more details about that
 214 [07:54] <@dholbach> did that scare you? :-)
 215 [07:54] <@dholbach> do you have any questions about the process?
 216 [07:55]  * jost86 dreams of being rich and famous ;)
 217 [07:55] <teknico> o/
 218 [07:55] <teknico> the pbuilder command built a .deb package
 219 [07:55] <@dholbach> teknico: so you're scared? or do you have a question? :)
 220 [07:55] <@dholbach> teknico: yep
 221 [07:55] <teknico> the dsc file is no more signed, though
 222 [07:56] <teknico> how come?
 223 [07:56] <rww> dholbach: "patch" in the above means debdiffs we create with https://wiki.ubuntu.com/PackagingGuide/Recipes/Debdiff , right?
 224 [07:57] <@dholbach> teknico: I usually don't use the .dsc files from *result/, but I guess it's to make sure you don't upload them accidentally
 225 [07:57] <imbrandon> teknico:  correct you'll have to source build and resign after pbuilder if testing goes ok
 226 [07:57] <@dholbach> teknico: dput will refuse to upload them for you (unless you force it to)
 227 [07:57] <imbrandon> rww: correct
 228 [07:57] <@dholbach> rww: a general patch should be fine, but a debdiff (with a nice changelog entry) will speed things up
 229 [07:58] <@dholbach> that's one of the most important points: always document your work properly
 230 [07:58] <@dholbach> next week we're going to have a session about PPAs, so Dustin Kirkland will talk a bit about how to upload packages to Launchpad
 231 [07:59] <@dholbach> that should be interesting and tie in quite well with this session
 232 [07:59] <@dholbach> any more questions? :)
 233 [07:59] <djails> can you point me to docs on how to package shared libs ?
 234 [07:59] <@dholbach> https://wiki.ubuntu.com/MOTU/GettingStarted should contain all the links I mentioned earlier
 235 [07:59] <@dholbach> djails: hang on
 236 [08:00] <@dholbach> http://www.debian.org/doc/debian-policy/ch-sharedlibs.html is the debian policy section for it
 237 [08:00] <@dholbach> and we had a session about that already, let me find it
 238 [08:00] <djails> thanks
 239 [08:00] <@dholbach> https://wiki.ubuntu.com/MOTU/School/LibraryPackaging
 240 [08:01] <djails> awesome !
 241 [08:01] <@dholbach> it's been a while since that session, but most things should not have changed
 242 [08:01] <@dholbach> also I'd recommend checking out a few very simple other library packages
 243 [08:01] <@dholbach> libsexy for example
 244 [08:01] <@dholbach> any other questions?
 245 [08:02] <sbeattie> dholbach: is there anything specific about packaging perl packages?
 246 [08:02] <@dholbach> ok cool - feel free to join #ubuntu-motu and ask all questions there - the people in there are really friendly and will help you out for sure
 247 [08:03] <@dholbach> sbeattie: I'm sure there is, I just did one in the past and used perlmodule.mk (part of CDBS) and made sure I used ${perl:Depends} somewhere
 248 [08:03] <@dholbach> sbeattie: that's all I know
 249 [08:04] <sbeattie> dholbach: okay, thanks.
 250 [08:05] <@dholbach> ok... thanks a lot everybody - it's been a great session!
 251 [08:05]  * Rail is going to request sponsorship for some packages :)
 252 [08:05] <@dholbach> I had a lot of fun and hope to see you guys around more often from now on!
 253 [08:05] <@dholbach> Rail: awesome!
 254 [08:05] <djails> dholbach: thanks, i ll try the REVU process for my pkg
 255 [08:05] <Rail> dholbach: thank you very much
 256 [08:05] <@dholbach> djails: perfect
 257 [08:06] <rww> dholbach: Are the various videos on the ubuntudevelopers youtube account still up-to-date (e.g. http://www.youtube.com/watch?v=zKLabbXTqMc ), or have there been any changes in the last year or so to watch out for?
 258 [08:06] <@dholbach> have a great day everybody and hope to see you at the next sessions too :)
 259 [08:06] <@dholbach> rww: there should be only small changes like change "hardy" to "jaunty" or something
 260 [08:06] <@dholbach> rww: in principle it should be all the same still
 261 [08:07] <rww> dholbach: excellent, thanks :)
 262 [08:07] <@dholbach> rock on everybody! :-)
 263 [08:07] <Crusher> thanks for the session dholbach, you have a great day aswell.
 264 [08:07] <jost86> thx
 265 [08:10]  * thekorn hugs dholbach, great session
 266 [08:10] <@dholbach> gracias thekorn
 267 [08:11] <dme> dholbach: Very useful, thanks.


Packaging/Training/Logs/2009-04-30-2 (last edited 2009-04-30 08:03:20 by i59F71706)