19:10:24 <coling> #startmeeting
19:10:24 <Inigo_Montoya> Meeting started Tue Feb  4 19:10:24 2014 UTC.  The chair is coling. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:10:24 <Inigo_Montoya> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:10:44 <coling> #topic i18n Automation for git+transifex
19:11:45 <yurchor> Personally, I think that now everything works fine. No additional efforts needed.
19:11:46 <coling> OK, so firstly, thanks to everyone for coping so well with the git migration. I appreciate some of the bits were not as convenient as previously and I had hoped to do more at the time, but sadly life+release etc etc.
19:11:53 <coling> yay!
19:12:35 <coling> Secondly, I'm not really up to speed with how things are working on transifex. I don't know how the integration works, so not really sure if there is more that can be done.
19:12:40 <filip__> coling: you did well!
19:12:52 <yurchor> +1
19:13:40 <coling> In the past we had talked about a synthetic repository that allowed easier direct access to all translations, but is this now no longer required with Transifex?
19:14:12 <filip__> In my opinion not.
19:14:19 <coling> [thanks! the next git migration will be even more complex!]
19:14:46 <coling> OK, cool. So are you guys actually happy with how things work? Is there anything manual that could be automated in the process?
19:14:54 <filip__> it would be more suitable if there would be a kind of synch between git and tx
19:14:55 <yurchor> coling: There is no need now. As soon as devs used to do make update_n_merge before the freeze and not to use voodoo...
19:15:31 <coling> yurchor, yeah so basically transifex becomes the canonical source of translations?
19:15:32 <filip__> another thing are emails on at least pot files commits
19:16:00 <coling> yurchor, e.g. if we change the .po files due to e.g. / removal etc., it doesn't flow to well to transifex?
19:16:20 <yurchor> coling: It can work both ways.
19:16:39 <coling> yurchor, OK, so what kind of voodoo specifically should be avoided?
19:16:50 <coling> (just so I understand!)
19:16:51 <yurchor> There is a simple set of command to make the synchronization.
19:17:06 <yurchor> Pot synchronization is done automatically.
19:17:40 <yurchor> coling: Thierry favorite sed magic.
19:18:14 <coling> yurchor, yeah but what kind of changes? e.g. if a message string changes only a little bit, it would seem sensible to update all po's to not break translations no?
19:18:21 * coling thought that was a good thing.
19:18:23 <yurchor> I mean without the need and as a replacement of automatic "make" extraction.
19:18:48 <yurchor> Humans tend to make the mistakes.
19:19:05 <coling> OK, so in this case, it's just a mistake rather than the principle?
19:19:41 <yurchor> It is just a guarantee that nothing will be forgotten.
19:19:43 <coling> e.g. if we do do some set magic, it would make sense to commit it, then run the update_n_merge just to make sure, even if you end up reverting it because it's just context changes?
19:19:50 <coling> s/set/sed/
19:19:55 <coling> ^^ irony
19:19:57 <coling> :)
19:20:14 <coling> s/reverting/resetting/
19:20:18 <coling> (before push)
19:20:29 <yurchor> Regretfully, there is no automatic way to find untranslateable and not extracted strings yet. :'(
19:20:46 <coling> Yeah not-extracted strings is always gonna be a problem really.
19:21:27 <coling> But OK, I think I've understood the problem in this case. sed-fu is not bad per-se, but we do need to be extra careful when doing it :)
19:21:49 <coling> So, is there something we can do to automatically do the git->transifex bit?
19:22:04 <Akien> I'm here, sorry for the delay
19:22:08 <Akien> Reading the backlog
19:22:10 <filip__> hi Akien
19:22:14 <coling> Hiya Akien
19:22:48 <yurchor> coling: As I said, it is enough to update POT with string changes regularly.
19:22:48 <filip__> it would be nice if po files would be synched in both ways
19:23:06 <yurchor> Hi Akien.
19:24:08 <coling> filip__, OK, so some kind of process that runs periodically and does the commits etc.?
19:24:22 <filip__> I think so
19:24:33 <yurchor> filip__: Yes... Nice to have. BTW GNOME Damned Lies now has an ability to commit translations to git with a button click from web interface...
19:24:37 <coling> filip__, have the scripts used currently (which presumably work manually) proven to be reliable?
19:25:09 <filip__> coling: which script exactly?
19:25:19 * coling doesn't know!
19:25:32 <coling> I mean whatever you do now to download+commit
19:25:41 <coling> (or upload)
19:25:48 <yurchor> coling: There is a problem with Latgalian and Uzbek Cyrillic so not everything can work automatically in both ways...
19:26:12 <coling> Ouch that's a shame.
19:26:13 <filip__> we now actually use git manually or tx. it depends on preference
19:26:32 <yurchor> tx push breaks on ltg and uz@cyrillic
19:27:14 <coling> yurchor, do you know if that's something that will be fixed at any point?
19:27:22 <coling> Or is it some kind of encoding issue?
19:27:43 <Akien> In my opinion, we should bypass uz@cyrillic and ltg for now, since we don't have translators for those locales
19:28:11 <yurchor> coling: I asked Tx guys twice. They even promise me to implement this in a week... Three months ago... :'(
19:28:34 <coling> yurchor, OK, so we likely shouldn't rely on it happening toooo soon!
19:28:52 <coling> But as Akien says we could blacklist those in any automatic integration.
19:29:59 <coling> OK, so can you guys point me to the workflow somewhere? e.g. how to log into tx etc. I'd guess we'd need to setup an account somewhere that can run those commands as/when needed.
19:30:46 <coling> FYI, I am hoping to deploy fedmsg in our infra at some point in the not too distant future. This is a federated message bus and it will act as the signals needed to trigger any automation.
19:30:47 <Akien> We don't need to sync them with tx as long as tx does not support them
19:30:47 <Akien> coling: Just to provide some context about what could be done for automation
19:32:41 <filip__> it seems there is a hickup on freenode
19:32:52 * yurchor is searching my old mails into i18n-discuss
19:33:12 <Akien> Yep, I was kicked for excess flood :)
19:34:18 <coling> Yeah it's been a bit flakey for the last day or so due to DDoS
19:35:00 <filip__> coling: what can be doen about commit emails part?
19:35:22 <coling> filip__, so I guess the problem is it doesn't preserve the original author of the translations?
19:36:03 <Akien> coling: It does now, but we have to do some work to put the authors list back into the po file and push them to Tx
19:36:06 <filip__> coling: ? I was asking about commit reports on 18n-reports@ml.mageia.org
19:36:24 <Akien> We lost the history when using our own Tx instance during the Mageia 1 release cycle
19:37:01 <coling> Akien, but we're good on that front ongoing?
19:37:07 <filip__> I mean somethink like atelier-commits@ml.mageia.org
19:37:12 <coling> filip__, ahh maybe I misnderstood :)
19:37:41 <filip__> it works great for web commits
19:37:59 <coling> I don't know whose emails.. lemme fine one to look at it.
19:38:12 <Akien> coling: As of now, new authors (i.e. accounts in Tx which edit a po file, or people added as a comment at the beginning of the po file) are saved and put in the po file as comments by Tx
19:38:24 <filip__> https://ml.mageia.org/l/arc/atelier-commits/2014-02/msg00060.html
19:38:26 <[mbot> [ atelier-commits - Commits on atelier repositories (Artwork, Web, etc ...) - arc_protect ]
19:39:13 <Akien> So copyright is no longer an issue for the future, but we should add the credits from Mandriva times and the Mageia 1, 2 and 3 release cycles
19:39:17 <yurchor> https://ml.mageia.org/wwsympa-wrapper.fcgi/arc/doc-discuss/2013-11/msg00066.html
19:39:18 <filip__> coling: better example: https://ml.mageia.org/l/arc/atelier-commits/2014-02/msg00063.html
19:39:18 <[mbot> [ doc-discuss - Discussions about Mageia documentation - arc_protect ]
19:39:19 <[mbot> [ atelier-commits - Commits on atelier repositories (Artwork, Web, etc ...) - arc_protect ]
19:40:25 <coling> Thanks for links. Regarding filip__ first... so commits to software do currently go to soft-commits
19:40:59 <coling> filip__, but I'm guessing you would quite like a subset of the commits (i.e. if the pot/po's are updated) sent also to the i18n-commits list?
19:41:12 <filip__> coling: yes
19:41:26 <filip__> and lang files as translation to web site
19:41:34 <coling> OK, that makes sense.
19:41:43 <filip__> and *.desktop also
19:41:47 <coling> I think it should be easy enough to do.
19:41:58 <Akien> coling: on i18n-reports ML maybe, since we already have it (and it's underused)
19:42:06 <filip__> +1
19:42:26 <coling> filip__, *.desktop? Shouldn't these be .desktop.in files and then extracted to pot like any other translation?
19:42:42 <coling> Akien, noted.
19:43:05 <coling> filip__, I know some of the soft repos do that currently (use .desktop.in files)
19:43:07 <filip__> not exactly yurchor created a script for that too. but I didn't manage to test it yet
19:43:25 <filip__> coling: let me find one
19:43:57 <coling> Ideally, translations should be kept exclusively in .po files for anything that is committed to the repo I should think (please correctly if I'm wrong or misunderstanding!)
19:44:35 <filip__> coling: http://gitweb.mageia.org/software/mageiawelcome/tree/usr/share/applications/mageiawelcome.desktop
19:44:36 <[mbot> [ mageiawelcome - Mageia Welcome Screen ]
19:44:54 <coling> filip__, yeah that one should definitely be a .desktop.in file I think.
19:44:56 <Akien> I'll be back in a few minutes
19:45:46 <filip__> coling: well. I don't know that but we have several such cases and they are in use
19:45:51 <coling> In the .desktop.in file the Name= line is replaces with an _Name= line which are extraced to the .pot and then intl-tool automatically converts it to a .desktop file on build and merged
19:46:25 <coling> filip__, Yeah, but I think it's technically wrong, so we should look to fix it.
19:46:33 <coling> lemme find a counter example.
19:47:02 <filip__> coling: https://wiki.mageia.org/en/Git_usage_for_l10n_and_doc#Software_resources_for_translation_in_our_git
19:47:44 <filip__> coling: we try to maintain that list
19:47:56 <coling> http://gitweb.mageia.org/software/control-center/tree/data/draksec.desktop.in
19:47:57 <[mbot> [ control-center - Mageia Control Center ]
19:48:37 <filip__> so you can see that there are: 1. po, 2. .desktop, 3. html, 4. lang files
19:49:24 <yurchor> coling: This needs intltool.
19:49:25 <coling> filip__, OK, but like I say intl-tool has a specific mode for .desktop files, so they this should really be corrected.
19:49:41 <coling> yurchor, yeah this can be applied at build time.
19:50:02 <coling> e.g. like done here: http://gitweb.mageia.org/software/control-center/tree/data/Makefile
19:50:03 <yurchor> coling: Some things are need not to be built...
19:50:03 <[mbot> [ control-center - Mageia Control Center ]
19:50:22 <yurchor> Just need to be packaged.
19:50:36 <filip__> coling: so what do you propose for .desktop files? bug reports?
19:51:00 <coling> filip__, I guess so yeah. depending how many there are I could likely fix them up fairly quickly.
19:51:04 <yurchor> ~30 messages... Bug reports... Pffff...
19:51:28 <yurchor> And not all these messages even used.
19:51:32 <coling> But yeah, separate bug reports seems overkill. Likely it'll just take someone to do it.
19:51:46 <coling> i.e. me probably ;)
19:51:51 <yurchor> We just need a general cleanup.
19:51:58 <filip__> +1
19:52:21 <filip__> and a correct and complete list
19:52:57 <coling> Technically we should really only run the intltool merge bit at tarball creation time, but our tarballing scripts are pretty rubbish (usually just a git-archive) so we just offload this to build time. It's easy enough so no big deal really.
19:53:19 <coling> OK, so... lemme create a couple of actions.
19:54:10 <coling> #action coling to look into having a commit notification email when certain files are updated (namely pot/po files but some others may be needed also (e.g. lang files for web pages)
19:54:37 <coling> Hmm, did that work?
19:54:43 <coling> #undo
19:54:43 <Inigo_Montoya> Removing item from minutes: <MeetBot.items.Action object at 0x8415d4c>
19:54:49 <coling> Seems so.
19:54:51 <coling> #action coling to look into having a commit notification email when certain files are updated (namely pot/po files but some others may be needed also (e.g. lang files for web pages)
19:55:43 <coling> #action catalog where direct translations are made to some files (e.g. .desktop files) and split into .in files and extract translations to .pot and merged in from .po
19:56:02 <coling> Seem sensible so far?
19:56:09 <filip__> coling: yes
19:56:22 <coling> OK, cool.
19:56:39 <filip__> coling: it would be nice to know for a new branch or new app for pot files too
19:56:59 <coling> It would likely be regexp based on the files altered.
19:57:12 <coling> So that _should_ happen automatically... in theory :)
19:57:16 <filip__> in svn there was svn list
19:57:40 <coling> git has ls-files
19:57:42 <filip__> but in git that is not easy for hundreds of repos
19:57:54 <coling> No :s
19:58:02 <filip__> ;)
19:58:48 <coling> Anyway, that should be fine for now, and we can add/adjust the regexp as needed to match more if needed.
19:59:25 <filip__> we created a script which helped us for that purpose: http://gitweb.mageia.org/software/i18n/tools/tree/check_for_translation_work.sh
19:59:27 <[mbot> [ tools - Tools for Managing Translations ]
20:00:03 <filip__> which basically svn/git up and searched for missing translation on soft and web
20:00:18 <coling> Phew! That's some script :)
20:00:22 <filip__> but it needs a list
20:00:36 <coling> So a list of repos?
20:00:45 <Akien> I'm back
20:00:49 <coling> Or a list of the files from each repo?
20:00:58 <filip__> like this http://gitweb.mageia.org/software/i18n/tools/tree/translation_projects.dat
20:01:24 <filip__> but I guess that should be done manually when we'll clear some things
20:01:30 <coling> Gotcha
20:01:58 <coling> Yeah, technically the repo list could be extracted from: http://gitweb.mageia.org/infrastructure/repositories/software/tree/
20:02:22 <coling> Essentially each *.repo file is what defines the git repos.
20:02:22 <filip__> the script used to do that in svn ;)
20:02:37 <filip__> but git is a complex beast ;)
20:03:03 <coling> Yeah, there are definitely some cons as well as pros (I can see how the cons affect i18n certainly!)
20:03:17 <filip__> great
20:03:34 <filip__> so what can be done about the list?
20:03:53 <[mbot> [ tools - Tools for Managing Translations ]
20:03:53 <[mbot> [ software - Software repository definitions ]
20:04:12 <coling> Well we could maintain some file somewhere I guess. i.e. a little db of repos + paths to po folders.
20:04:27 <coling> And have it automatically update on push.
20:04:35 <filip__> that would be great
20:04:50 <filip__> but we need support from devs on that too
20:05:06 <filip__> about cleaning
20:05:10 <coling> In what way would it need dev support?
20:05:15 <coling> (other than doing it in the first place?)
20:05:36 <filip__> when clean I guess it should work
20:05:50 <coling> yeah in an ideal world it would "just happen"
20:06:03 <coling> Assuming the "po" folder naming convention.
20:06:30 <coling> So what should the file look like?
20:06:43 <filip__> you mean the list?
20:06:45 <coling> repopath pofolderpath
20:06:54 <coling> Yeah.
20:07:15 <coling> Then it could be grabbed via some URL, and you could parse it and use it in the script.
20:07:17 <filip__> if you create a csv I guess I can build a web report
20:07:40 <coling> Sure so something like
20:07:44 <coling> control-center po
20:07:55 <coling> drakx install/po
20:08:02 <coling> drakx install/standalone/po
20:08:05 <coling> etc.
20:08:10 <coling> (but with commas)
20:08:12 <coling> :)
20:08:32 <coling> So first field is the repo path (so they should have "software/" prefixed actually
20:08:39 <filip__> repo comma path name of pot file
20:08:39 <coling> and the second the path inside the repo.
20:08:52 <coling> OK, so pot file is the best thing to search.
20:09:07 <filip__> yeah. but not the only
20:09:13 <filip__> see other 3 up
20:09:16 <Akien> Yes pot file would be needed I guess, since its name can be anything
20:09:59 <filip__> then we can do something like: http://www.mageia.org/langs/report.php
20:10:06 <[mbot> [ www.mageia.org translation report ]
20:10:13 <coling> so for drakx it would be:
20:10:16 <coling> software/drakx,perl-install/install/help/po/DrakX-help.pot
20:10:16 <coling> software/drakx,perl-install/install/share/po/DrakX.pot
20:10:16 <coling> software/drakx,perl-install/standalone/po/libDrakX-standalone.pot
20:10:16 <coling> software/drakx,perl-install/share/po/libDrakX.pot
20:10:40 <filip__> make it simple
20:10:46 <filip__> we can fix it latter
20:10:53 <filip__> ;)
20:11:19 <coling> Yup, so above would be OK for you yeah?
20:11:35 <coling> Do we need to include branches?
20:11:42 <filip__> for start it would be more than great
20:11:54 <coling> e.g. repo,branch,potpath
20:11:54 <filip__> I'll try to do some php magic then
20:12:19 <filip__> maybe add pot filename
20:12:53 <coling> OK, then:
20:12:55 <Akien> filip__: Maybe that:
20:12:58 <Akien> software/drakx,perl-install/install/help/po,DrakX-help.pot
20:13:03 <filip__> but it would be nice if active branches would be there too
20:13:14 <filip__> sounds perfect
20:13:41 <Akien> Then you don't have to use some magic to extract the path without the pot file to access the po files :)
20:13:42 <coling> #action coling to work on a script that that extracts "reponame,branch,potfile" from repos and publishes to a central location.
20:13:59 <filip__> \o/
20:14:18 <coling> Akien, basename and dirname in scripts is pretty easy. But I could do that on the server side at generation time if it's useful.
20:14:39 <filip__> and .desktop* files too.
20:15:22 <coling> filip__, I don't think we'd need to care about .desktop files if they were converted to .in files. They'd just be the same as anything other translatable file then - no need for special handling.
20:15:24 <filip__> a list would be great
20:15:41 <coling> filip__, but I could script a on-time extraction of them if you like?
20:15:52 <coling> (for the purposes of looking into converting them to .in files)
20:15:56 <filip__> sorry I don't follow
20:16:27 <filip__> so those  .desktop files will be converted to pot files?
20:16:38 <Akien> filip__: IIUC, the translatable bits from the .desktop.in files will be extracted to the main pot file of the project
20:16:52 <coling> filip__, yeah what Akien said.
20:16:53 <filip__> that would be awesome
20:16:57 <Akien> So we won't have to handle them separately
20:17:05 <coling> filip__, this is how it works currently in several repos
20:17:10 <coling> filip__, e.g. control-center
20:17:21 <filip__> yurchor created a scripts for that too: http://gitweb.mageia.org/software/i18n/tools/tree/desktop
20:17:23 <[mbot> [ tools - Tools for Managing Translations ]
20:18:25 <coling> filip__, I think that script would ultimately no longer be needed if things were converted.
20:18:29 <filip__> I'll be happy if we get rid of l10n of .desktop files in any form ;)
20:18:42 <coling> filip__, see here: http://gitweb.mageia.org/software/control-center/tree/data
20:18:43 <[mbot> [ control-center - Mageia Control Center ]
20:18:48 <coling> The files are all .in files.
20:19:24 <coling> Theese files are all added automatically to POTFILES.in here: http://gitweb.mageia.org/software/control-center/tree/po/Makefile
20:19:25 <[mbot> [ control-center - Mageia Control Center ]
20:19:50 <coling> And thus the strings are extracted to the pot file.
20:20:08 <coling> Then then .desktop file is generated at build time and merges in the translations from .po files here: http://gitweb.mageia.org/software/control-center/tree/data/Makefile
20:20:09 <[mbot> [ control-center - Mageia Control Center ]
20:20:30 <coling> filip__, so yeah if we can get a one-time list of .desktop files I can look to convert to this scheme.
20:20:43 <filip__> thx for clarification. my missunderstanding worsen situation a bit
20:21:08 <coling> #action coling to look into conversion for static .desktop files to translatable .desktop.in files for easier handling.
20:21:28 <filip__> the list is here https://wiki.mageia.org/en/Git_usage_for_l10n_and_doc#Software_resources_for_translation_in_our_git
20:21:54 <coling> Ahh gotcha
20:22:04 <coling> I didnd't see the specific section last time!
20:22:18 <filip__> what will happen with existing translations in .desktop files
20:22:28 <coling> Well..... ;)
20:22:41 <coling> I could use some sed magic to add them to the appriate .po files.
20:23:03 <coling> But I'll make sure this is valid before pushing so as not to make yurchor cry :p)
20:23:07 <Akien> We definitely a wiki.mageia.org/en/Sed_magic :D
20:23:17 <filip__> #link  https://wiki.mageia.org/en/Git_usage_for_l10n_and_doc#Software_resources_for_translation_in_our_git
20:23:51 <filip__> Akien: I would need it sometimes ;)
20:24:02 <coling> OK, so that's a few actions so far. I think these things can all be done with current git hooks infra. For further integrations I think I'd want to wait until we have fedmsg
20:24:16 <yurchor> coling: Thanks. I will not cry but thousands of little kitties will do. ;)
20:24:22 <coling> :p
20:24:55 <coling> The other part is that I'd like to finish off and do remaining git conversions for e.g. web stuff.
20:25:25 <coling> It should be fairly straight forward but obviously it'll affect you guys. I won't do that immeidately tho'. I'll let the distro settle for a couple weeks.
20:25:40 <coling> That OK?
20:25:46 <filip__> coling: what we conclude for tx <-> git synch of po files?
20:26:10 <coling> filip__, I think it could be done, but for things which involved automatically committing, I'd like to wait until we have fedmsg.
20:26:29 <filip__> fedmsg?
20:26:30 <coling> filip__, we could do something via cron but it's not as nice.
20:27:10 <coling> filip__, basically it's a federated message bus that lets us send signals when various events occur, e.g. a push, a bug report opened, a package built etc. etc. (lots fo things)
20:27:39 <coling> You can then subscribe to those messages from various little daemons and react accordingly
20:27:40 <filip__> any time frame for that?
20:27:52 <filip__> mga5?
20:27:56 <coling> filip__, soon ish.
20:28:04 <filip__> great
20:28:11 <coling> It's not related to mga release, so we just need to package it up.
20:28:17 <filip__> yeah
20:28:20 <coling> We can do a lot of cool things with it.
20:28:44 <coling> I'm going to try and bribe a python guru into packaging it and its deps.
20:29:06 <filip__> there are currently 2 html resources. one: http://gitweb.mageia.org/software/design/bootloader-theme/tree/help-boot/en
20:29:08 <[mbot> [ bootloader-theme - Mageia Bootloader Theme ]
20:29:18 <filip__> any idea on that?
20:30:49 <coling> Not off the top of my head, but I'd suggest it *should* really be pot'ified in some capacity.
20:31:00 <coling> But the best way to do that, I'm not sure.
20:31:16 <filip__> tx can handle it for now. maybe we shoul push it there
20:31:23 <Akien> Actually I'm not even sure those resources are still relevant
20:31:26 <Akien> Technically speaking I mean
20:31:36 <filip__> they are used in boot help
20:31:47 <filip__> but obsolete
20:31:54 <filip__> in some degree
20:32:32 <filip__> coling: we filled your cup ;)
20:32:47 <filip__> thx for your dedication from my side
20:33:02 <coling> Yeah I think I've got a few jobs to crack on with for now. We can revisit the automated magic to tx once we have fedmsg
20:33:03 <filip__> yurchor: Akien: anything to add?
20:33:05 <coling> :)
20:33:20 <yurchor> No.
20:33:44 <coling> OK, let's call it a night for now then. :)
20:33:48 <coling> #endmeeting