| Wizzup | freemangordon: ok, tried that now, let's see if this works out | 00:32 |
|---|---|---|
| Wizzup | I see this btw: | 00:33 |
| Wizzup | $ tracker daemon -s | 00:33 |
| Wizzup | Starting miners… | 00:33 |
| Wizzup | (tracker daemon:30492): Tracker-CRITICAL **: 00:32:48.217: Could not create proxy on the D-Bus session bus, Error calling StartServiceByName for org.freedesktop.Tracker1.Miner.Extract: Timeout was reached | 00:33 |
| Wizzup | freemangordon: there is also .cache/tracker btw | 00:38 |
| Wizzup | freemangordon: so I get the exact problem after I killed the processes, did a hard reset, removed any extra files | 02:06 |
| Wizzup | same problems: tracker-extract starting constantly | 02:06 |
| Wizzup | with was with mafw not even running | 02:07 |
| Wizzup | this was* | 02:07 |
| freemangordon | Wizzup: maybe try to do hard reset, remove both directories (.cache/tracker and .local/share/tracker/) and then start trackers | 07:29 |
| freemangordon | if it still does not work, I am out of ideas | 07:29 |
| freemangordon | and yes, mafw has nothing to do | 07:29 |
| arno11 | is it normal that my device (up-to-date) is not affected by trackers issue ? | 09:15 |
| arno11 | just to be sure | 09:16 |
| freemangordon | seems the issue appears when there is a huge amount of music | 09:18 |
| arno11 | ok | 09:18 |
| freemangordon | also, after the last experiments, my device does not suffer as well | 09:19 |
| arno11 | ah cool | 09:19 |
| freemangordon | not really, as now i can't repro :) | 09:19 |
| arno11 | ok :) | 09:20 |
| Wizzup | freemangordon: I did that | 12:31 |
| freemangordon | :( | 14:34 |
| freemangordon | Wizzup: how many GB of music do you have? | 14:35 |
| Wizzup | let me check | 14:37 |
| Wizzup | # du -sh /mnt/sd/music/ | 14:37 |
| Wizzup | 17G/mnt/sd/music/ | 14:37 |
| freemangordon | ok | 14:37 |
| freemangordon | I'll copy some more to the VM | 14:37 |
| Wizzup | I wonder if we need to take a step back to try to figure out what's going on here | 14:38 |
| Wizzup | this seems like such a grave tracker bug maybe something else is going on | 14:38 |
| freemangordon | mhm | 14:38 |
| freemangordon | for sure thre is a nasty memleak in mafw-tracker-source | 14:38 |
| freemangordon | I already fixed, have to issue a build | 14:38 |
| Wizzup | it looks like the tracker irc channel is mostly empty, probably the irc bridge is gone, so we'll have to install a matrix client to talk to gnome | 14:39 |
| Wizzup | well, unless valgrind finds serious problems (not just with memleaks), I doubt they can cause the issues we are seeing | 14:39 |
| Wizzup | so you are sure that the behaviour we are seeing is because one db is deleted, and the other doesn't realise, and the db gets deleted, why? do we know? | 14:39 |
| freemangordon | yes, db gets deleted because tracker receives "CORRUPTED" result from some sqlite query | 14:40 |
| freemangordon | but I can;t repro anyhmre | 14:41 |
| Wizzup | and then they just decide to the delete the db? interesting | 14:41 |
| freemangordon | on both my d4 and my VM after I deletet *all* tracker related files, I see no issue anymore | 14:41 |
| Wizzup | I mean I'm happy to do it again, I just don't think it'll make a diff, sec | 14:42 |
| freemangordon | so: | 14:42 |
| freemangordon | stop ,mafw | 14:42 |
| freemangordon | tracker reset -r | 14:42 |
| Wizzup | mafw-source-tracker is stopped | 14:42 |
| freemangordon | start mafw | 14:42 |
| freemangordon | oh,sorry | 14:42 |
| freemangordon | tracker reset -r | 14:42 |
| Wizzup | I am confused now | 14:43 |
| freemangordon | rm -rf ~/.cache/tracker | 14:43 |
| freemangordon | rm -rf ~/.local/share/tracker/ | 14:43 |
| freemangordon | start mafw | 14:43 |
| freemangordon | why? | 14:43 |
| Wizzup | and then tracker daemon -s | 14:43 |
| Wizzup | ? | 14:43 |
| freemangordon | no need | 14:44 |
| freemangordon | starting mafw will start the tracker | 14:44 |
| freemangordon | why are you confused? | 14:44 |
| Wizzup | I was confused when you wrote 'oh,sorry' | 14:44 |
| Wizzup | in any case I did what you wrote above | 14:44 |
| Wizzup | the only difference is that I didn't even run mafw, I just ran tracker daemon -s | 14:44 |
| freemangordon | because I missed the 'rm -rf' steps :) | 14:44 |
| Wizzup | to rule out mafw messing anything up | 14:44 |
| freemangordon | I see | 14:45 |
| freemangordon | anyway, I have 11GB on my VM | 14:45 |
| freemangordon | the diff is that it is in MyDocs | 14:45 |
| freemangordon | maybe that's the reason | 14:45 |
| Wizzup | does mafw-dbus-wrapper honour /etc/profile.d btw? | 14:45 |
| freemangordon | no idea | 14:46 |
| freemangordon | what is in there? | 14:46 |
| Wizzup | looks like it | 14:46 |
| Wizzup | $ cat /etc/profile.d/tracker-maemo.sh | 14:46 |
| Wizzup | export TRACKER_USE_LOG_FILES=1 | 14:46 |
| Wizzup | freemangordon: ^ | 14:46 |
| freemangordon | ah | 14:46 |
| Wizzup | well so far only tracker-store is running, but maybe others will start eventually | 14:46 |
| freemangordon | I am not sure what happens if started it from xsession file | 14:46 |
| freemangordon | yes, they will | 14:47 |
| freemangordon | I'll do the same here | 14:47 |
| freemangordon | to see if it will fail this time | 14:47 |
| freemangordon | (in the VM) | 14:47 |
| Wizzup | right | 14:47 |
| freemangordon | btw, how do you stop mafw? | 14:47 |
| Wizzup | /usr/bin/mafw.sh stop mafw-tracker-source | 14:47 |
| freemangordon | ok | 14:48 |
| Wizzup | still only tracker-store | 14:49 |
| Wizzup | 6 mins later | 14:49 |
| freemangordon | yeah | 14:49 |
| Wizzup | shall I run tracker daemon -s ? | 14:49 |
| freemangordon | no | 14:49 |
| Wizzup | when/how do these get started then? | 14:50 |
| freemangordon | I guess tracker-store uses lots of cpu | 14:50 |
| Wizzup | it's doing something still | 14:50 |
| freemangordon | tracker daemon -f | 14:50 |
| Wizzup | it just finished | 14:50 |
| freemangordon | and? | 14:51 |
| freemangordon | what does tracker daemon -f say? | 14:52 |
| Wizzup | freemangordon: following | 14:58 |
| Wizzup | it's crawling various dirs | 14:58 |
| Wizzup | and now is processing | 14:58 |
| Wizzup | extractor not running yet | 14:58 |
| freemangordon | ok | 14:59 |
| freemangordon | hmm, why extractor is not running? | 14:59 |
| freemangordon | it should | 14:59 |
| freemangordon | ah, wait | 15:00 |
| freemangordon | miner shall finish with crawling first | 15:00 |
| Wizzup | yes | 15:00 |
| Wizzup | now it is running on my d4 | 15:00 |
| Wizzup | brb | 15:00 |
| freemangordon | ok, after copying some more music in the VM, it happened again | 15:11 |
| freemangordon | Wizzup: "(tracker-store:7084): Tracker-CRITICAL **: 16:49:35.135: SQLite error: database disk image is malformed (errno: Success)" | 15:50 |
| Wizzup | do we have a line number for this somewhere? | 15:53 |
| freemangordon | rebuilding tracker with debug symbols, will have it in few minutes | 15:54 |
| freemangordon | Wizzup: ../src/libtracker-data/tracker-db-interface-sqlite.c:2346 | 15:57 |
| freemangordon | this is where g_unlink is called from | 15:57 |
| freemangordon | so it comes from link 2341 | 15:58 |
| Wizzup | ok, I need to get vm started and apt-get source | 15:59 |
| freemangordon | lemme run that under valgrind | 15:59 |
| Wizzup | ah, you got it to the point where it doesn't actually remove the db? that's good | 16:00 |
| Wizzup | (for testing) | 16:00 |
| freemangordon | no | 16:00 |
| freemangordon | I put a BP in g_unlink :) | 16:00 |
| freemangordon | and it got hit right after that error | 16:00 |
| freemangordon | so it does remove the db | 16:00 |
| freemangordon | so, somehow, someone is corrupting the database | 16:01 |
| freemangordon | and that seems to be for real | 16:01 |
| Wizzup | SQLITE_CORRUPT isn't as bad as it sounds from my reading | 16:03 |
| Wizzup | but yes, something is weird there | 16:03 |
| freemangordon | it should not happen without a crash, no? | 16:03 |
| freemangordon | btw, there are 3453 frames in the backtrace | 16:04 |
| freemangordon | it seems to be executing huge query | 16:05 |
| freemangordon | hundreds if not thousands of DELETE statements | 16:06 |
| Wizzup | not sure why there should be so many frames if it's a big delete | 16:07 |
| freemangordon | I suspect this is the final cleanup | 16:07 |
| freemangordon | maybe they do a recursion | 16:07 |
| Wizzup | *facedesk* :) | 16:08 |
| freemangordon | yes, I see 2 functions again and again | 16:08 |
| Wizzup | Maybe it runs out of stack depth then or something? | 16:08 |
| Wizzup | it might also be that it runs out of the size limits they themselves set | 16:09 |
| Wizzup | but still, this should happen to many people, no? | 16:09 |
| freemangordon | ok, under valgrind there is no issue :( | 16:09 |
| freemangordon | no idea | 16:09 |
| freemangordon | lemme try again | 16:09 |
| Wizzup | I don't know if the page_size or cache_size have any effect here | 16:12 |
| Wizzup | it looks like I've gotten into the same situation again btw | 16:12 |
| Wizzup | last update was 08 Dec 2023, 16:05:19: 69% Extractor - Extracting metadata 05m 55s remaining | 16:12 |
| Wizzup | and now the extractor is always idle | 16:12 |
| Wizzup | and the db 'seems' empty | 16:12 |
| Wizzup | interesting, my tracker-miner-fs has two db handles open it looks like | 16:13 |
| Wizzup | at least judging from it's fds | 16:13 |
| Wizzup | 0 lr-x------ 1 user user 64 Dec 8 16:12 10 -> '/home/user/.cache/tracker/meta.db (deleted)' | 16:13 |
| Wizzup | 0 lrwx------ 1 user user 64 Dec 8 16:12 11 -> '/home/user/.cache/tracker/meta.db-wal (deleted)' | 16:13 |
| Wizzup | 0 lrwx------ 1 user user 64 Dec 8 16:12 12 -> '/home/user/.cache/tracker/meta.db-shm (deleted)' | 16:14 |
| Wizzup | 0 l-wx------ 1 user user 64 Dec 8 16:12 13 -> /home/user/.local/share/tracker/data/tracker-store.journal | 16:14 |
| Wizzup | 0 lr-x------ 1 user user 64 Dec 8 16:12 14 -> anon_inode:inotify | 16:14 |
| Wizzup | 0 lr-x------ 1 user user 64 Dec 8 16:12 15 -> '/home/user/.cache/tracker/meta.db (deleted)' | 16:14 |
| Wizzup | 0 lrwx------ 1 user user 64 Dec 8 16:12 16 -> '/home/user/.cache/tracker/meta.db-wal (deleted)' | 16:14 |
| Wizzup | looks like the same is true for tracker-extract, so maybe that means nothing | 16:14 |
| Wizzup | (but the ones it looks at aren't deleted of course) | 16:15 |
| freemangordon | it means that it was tracker-extract to delete the db | 16:17 |
| Wizzup | it gets started every 15 seconds | 16:18 |
| Wizzup | so I am not sure if that is true | 16:18 |
| Wizzup | but your logs should tell you | 16:18 |
| Wizzup | btw, I asked in the #gnome-tracker channel here just in case there would be a response | 16:18 |
| Wizzup | https://www.sqlite.org/forum/forumpost/2f005babea?t=c&unf | 16:20 |
| Wizzup | not sure if this is relevant, googling atm for 'SQLITE_CORRUPT gnome tracker' | 16:20 |
| Wizzup | we do have this specific version | 16:20 |
| Wizzup | (of sqlite3) | 16:21 |
| Wizzup | https://gitlab.gnome.org/GNOME/tracker/-/merge_requests/353 | 16:21 |
| Wizzup | do we have this? | 16:21 |
| Wizzup | freemangordon: ^^ | 16:21 |
| freemangordon | I doubt | 16:22 |
| Wizzup | well, we have 3.34.1-3 | 16:22 |
| Wizzup | and I have text files in my music | 16:22 |
| freemangordon | mhm | 16:22 |
| freemangordon | lemme pull that and retest | 16:22 |
| Wizzup | btw the dev seems active https://discourse.gnome.org/t/tracker-reports-database-disk-image-is-malformed/17344 | 16:29 |
| Wizzup | he's also on irc but didn't respond yet | 16:29 |
| Wizzup | he just did | 16:32 |
| Wizzup | said 2.x is stagnant and not maintained, so that the problem is probably with updated deps (sqlite3), and that it would be best to switch to 3.x if we can | 16:33 |
| Wizzup | 3.6.x specifically | 16:33 |
| Wizzup | freemangordon: I'm getting a bit more confident this really is our problem | 16:33 |
| freemangordon | mhm | 16:34 |
| freemangordon | testing atm | 16:34 |
| freemangordon | with the fix that is | 16:34 |
| freemangordon | seems like | 16:35 |
| freemangordon | lemme clone tracker | 16:35 |
| Wizzup | freemangordon: btw, want to join #gnome-tracker even just to idle? the dev is somewhat chatty and then I won't have to relay the info | 16:36 |
| Wizzup | he's linking me a list of changes from tracker 2-3, which you're probably aware of already (https://gnome.pages.gitlab.gnome.org/tracker/migrating-2to3.html) | 16:36 |
| freemangordon | Wizzup: in the meanwhile, could you add tracker to jenkins? | 17:05 |
| Wizzup | sure, I thought we already had it | 17:07 |
| Wizzup | I guess we had miners? | 17:07 |
| Wizzup | freemangordon: what repo name? | 17:07 |
| freemangordon | tracker | 17:08 |
| freemangordon | miners are separate repo | 17:08 |
| Wizzup | added | 17:08 |
| freemangordon | https://github.com/maemo-leste-upstream-forks/tracker | 17:08 |
| freemangordon | ok, thanks | 17:08 |
| freemangordon | Wizzup: what?!? https://phoenix.maemo.org/job/tracker-source/1/consoleText | 17:13 |
| freemangordon | fatal: Authentication failed for 'https://github.com/maemo-leste-upstream-forks/tracker/' | 17:13 |
| Wizzup | freemangordon: hangon | 17:14 |
| freemangordon | oh, wait | 17:14 |
| freemangordon | my bad | 17:15 |
| Wizzup | freemangordon: the repo is private | 17:15 |
| Wizzup | want me to fix? | 17:15 |
| freemangordon | fixed | 17:15 |
| freemangordon | why private is the default? | 17:15 |
| Wizzup | github :) | 17:15 |
| Wizzup | idk | 17:15 |
| Wizzup | they hate sharing I guess | 17:15 |
| freemangordon | M$ you mean :) | 17:16 |
| Wizzup | yes | 17:16 |
| freemangordon | Wizzup: please upgrade tracker to see if the issue is fixed for you | 17:28 |
| freemangordon | hmm with the new tracker 11GB were indexed in less than a minute | 17:33 |
| freemangordon | on d4 that is | 17:33 |
| Wizzup | let me try | 17:35 |
| freemangordon | i just reset the db | 17:36 |
| freemangordon | and restarted mafw | 17:36 |
| freemangordon | no manual deletion etc | 17:36 |
| Wizzup | did you restart mafw? | 17:37 |
| freemangordon | (18,36,09) freemangordon: and restarted mafw | 17:37 |
| Wizzup | ah | 17:37 |
| Wizzup | ha | 17:37 |
| Wizzup | ok | 17:37 |
| freemangordon | actually after db reset, I just killed mafw-tracker-source | 17:37 |
| freemangordon | that's enough to trigger a new index | 17:37 |
| Wizzup | doesn't look like mine will finish in a minute | 17:41 |
| freemangordon | well... | 17:42 |
| Wizzup | but hey, as long as it works :D | 17:42 |
| freemangordon | you'd better pray for it to finish at all :p | 17:42 |
| Wizzup | I'm not a religious man, but I'll sacrifice some plant | 17:42 |
| freemangordon | me neither, but that does not mean we shall lose opportunities... what is... :) | 17:43 |
| freemangordon | *what if | 17:43 |
| Wizzup | actually it is going much faster now it seems | 17:44 |
| freemangordon | mhm | 17:44 |
| Wizzup | 08 Dec 2023, 17:43:41: 35% File System - Processing… 02m 55s remaining | 17:44 |
| Wizzup | maybe some optimisations from our CI? | 17:44 |
| freemangordon | I doubt | 17:44 |
| freemangordon | but... who knows | 17:44 |
| Wizzup | hah, now omp shows the right time indication as well | 17:45 |
| Wizzup | it follows tracker daemon -f | 17:45 |
| freemangordon | yes | 17:45 |
| freemangordon | ok, if that's fixed, during the weekend I'll put xv support back in mafw-gst-renderer | 17:47 |
| freemangordon | after that we can push all the recent work on multimedia to -stable | 17:47 |
| Wizzup | cool | 17:49 |
| Wizzup | agreed | 17:49 |
| Wizzup | let's monitor pm meanwhile | 17:49 |
| freemangordon | yeah | 17:50 |
| Wizzup | I could probably use some help with telepathy on the sip/xmpp front, in debugging some issues | 17:50 |
| freemangordon | sure | 17:50 |
| freemangordon | did it finish? | 17:50 |
| freemangordon | (tracker) | 17:50 |
| Wizzup | still extracting | 17:52 |
| Wizzup | probably 5 more mins | 17:52 |
| freemangordon | 5 minutes passed :) | 18:03 |
| Wizzup | 13:21 eta | 18:04 |
| freemangordon | omg | 18:04 |
| Wizzup | my files are complicated :) | 18:04 |
| Wizzup | btw, there is telepathy-tank | 18:04 |
| Wizzup | we have it packaged I think (I packaged it) | 18:04 |
| Wizzup | but I didn't get it to work yet with conversations | 18:04 |
| Wizzup | https://github.com/TelepathyIM/telepathy-tank | 18:04 |
| freemangordon | we will need accounts plugin I guess | 18:05 |
| Wizzup | well there was more to it | 18:06 |
| Wizzup | as in, I got the account set up | 18:09 |
| Wizzup | but I only got it to connect once iirc | 18:09 |
| Wizzup | 4 mins left | 18:19 |
| Wizzup | eta is a bit off:p | 18:19 |
| freemangordon | I wonder why it is so slow on your device | 18:20 |
| freemangordon | how complicated could a music be? | 18:20 |
| Wizzup | I don't know where you get it from, but I ripped a lot of it from CDs using custom sw that adds all kinds of metadata and embeds images in the files | 18:20 |
| Wizzup | some of it is flac too | 18:20 |
| Wizzup | 08 Dec 2023, 18:20:39: ✓ Extractor - Idle | 18:21 |
| Wizzup | let's see | 18:21 |
| Wizzup | brb, have to make some dinner | 18:22 |
| Wizzup | looks like tracker is pacified for now | 18:46 |
| freemangordon | yeah | 18:53 |
| Wizzup | btw, I took a quick look earlier today what would 'break' without python2 and pygtk2 and hildon pkgs in python | 20:22 |
| Wizzup | these packages will not work anymore: binaryclock-plugin, quicknote, sojourner, syncevolution-frontend, modrana | 20:22 |
| Wizzup | from extras | 20:22 |
| Wizzup | which depend on one or more of these: | 20:23 |
| Wizzup | python2-gconf python-conic python-hildon python-hildondesktop python-location python-mafw python-osso | 20:23 |
| freemangordon | Wizzup: did you unpack your PPP? | 20:47 |
| Wizzup | freemangordon: no, but I can tomorrow, or in ~2 hours or so | 20:48 |
| freemangordon | ok, lets try to find some time tomorrow to look at h-d issue | 20:48 |
| freemangordon | that's pointed to me as well | 20:49 |
| Wizzup | ok | 21:02 |
Generated by irclog2html.py 2.17.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!