Session Start (sean_nothings:muratorigo): Mon Apr 02 02:09:18 2007
[02:09] *** (Link: yahoo://0:yahoo-profile:muratorigo)muratorigo has added you to their contact list with reason: "Looks like Miranda IM still has all my old chat accounts working somehow... 
".  You may choose to (Link: yahoo://0:yahoo-accept:muratorigo)accept or (Link: yahoo://0:yahoo-deny:muratorigo)deny this action.  You may also (Link: yahoo://0:yahoo-add:muratorigo)add this user to your contact list or (Link: yahoo://0:yahoo-ignore:muratorigo)ignore this user.
[02:09] muratorigo: Hello?>
[02:09] muratorigo: (typotastic)
[02:09] sean_nothings: yo
[02:09] *** You have been successfully added to muratorigo's contact list.
[02:10] sean_nothings: hang on, I'm cleaning up windows
[02:10] muratorigo: Alright duder.
[02:10] muratorigo: I'm going to go download the latest Vorbis source.
[02:10] sean_nothings: is your existing thing old?
[02:10] sean_nothings: but ok
[02:11] muratorigo: Well, I don't want to use the existing one because I modified it.
[02:11] sean_nothings: (Wait, you called the gmail chat a POS. I thought you thought it was awesome.)
[02:11] sean_nothings: Sure
[02:11] muratorigo: I'd rather use the fresh source to make sure nothing I did is affecting the logs
[02:11] sean_nothings: Do you have the IGJ4 tree?
[02:11] muratorigo: Thought is was awesome?
[02:12] muratorigo: Not at all.
[02:12] muratorigo: But I don't think this one is awesome either :)
[02:12] sean_nothings: Well, you seemed to think it was a fine client compared to 'real'... ah, ok
[02:12] muratorigo: So I was trying to figure out why you wanted to switch :)
[02:12] sean_nothings: are you using yahoo, or trillian?
[02:12] muratorigo: Miranda.
[02:12] sean_nothings: not that trillian is _awesome_, but
[02:12] sean_nothings: tolerability higher, anyway
[02:12] muratorigo: And like, Miranda from 1902 since I don't ever use chat, so it's still the install from when I tried out chat.
[02:13] sean_nothings: nice
[02:13] muratorigo: OK, do you want me to use the vanilla source or the IGJ4 source then?
[02:13] muratorigo: I assume vanilla source would be the least problematic.
[02:13] sean_nothings: if you want to start from scratch, that's fine
[02:13] sean_nothings: I was just thinking it's already built and there's a sample app in the IGJ4 tree if you want a quick start
[02:14] muratorigo: I've already hacked through this thing once, so it shouldn't be a problem.
[02:14] sean_nothings: ok
[02:14] muratorigo: I'm assuming since this is a clean-room thing, I also mustn't ever mention anything about the source, either?
[02:15] sean_nothings: you'll need vorbis and vorbisfile and ogg, I guess, but the logging is probably only in vorbis
[02:15] sean_nothings: yeah
[02:15] muratorigo: Roger that.
[02:15] sean_nothings: well, Jeff was willing to let me go instrument the source myself before I proposed this, so if a _little_ leaks out it's probably fine, but better safe than sorry
[02:15] sean_nothings: so, like, you can say "I can't find anything like that" and that's a little leak-y but probably ok (but if we can get by without that somehow that would be cool too)
[02:16] muratorigo: I think we should be able to do the whole thing without me ever saying anything explicit, because I know roughly how it works, so there shouldn't be any backchannel.  You can just say "I need the x", and I'll go get the x.
[02:16] sean_nothings: basically it's just a question of how far they deviate from the spec structurally or in optimizing (deferring things, combining phases, etc). If they've done that, then that's a step we can't log anything from, and we move on. If I'm really unlucky, there won't be anything, but I doubt it
[02:17] muratorigo: OK, give me a minute to get the vanilla source building.  You got a particular OGG file that we're going to use as the gold sample?
[02:17] sean_nothings: Anyway, let me know when you're ready to start. I dunno if you want to get a sample app building first or what
[02:17] sean_nothings: I have a couple
[02:17] sean_nothings: Let me upload them
[02:17] muratorigo: OK, let's get started on getting those me while I get the source building.  I'll check back in a bit.
[02:22] sean_nothings: whoa, I think my upload bandwidth has been raised, I'm getting 43KB/s instead of 30
[02:22] sean_nothings: stupid ADSL
[02:22] muratorigo: Are we doing encoding, or just decoding?
[02:22] sean_nothings: just decodiong
[02:22] muratorigo: Roger that.
[02:22] sean_nothings: upload'll be done in two minutes
[02:23] sean_nothings: and I've verified that there's a log being made, yay
[02:23] sean_nothings: do you think I should edit out the random chatter from the log, or does that compromise its trustworthiness?
[02:24] muratorigo: I would assume you would want the log exactly as it.
[02:24] sean_nothings: (Link: http://nothings.org/misc/oggtest.zip)http://nothings.org/misc/oggtest.zip
[02:24] muratorigo: (as it is... sorry typo again)
[02:24] sean_nothings: there's a bunch in that, I'm assuming it's not going to take any time to do 5 instead of 1, and better to get it all now then run into something later
[02:25] muratorigo: Good point.
[02:25] muratorigo: OK, so do you want to start typing in an overview of what we're doing here?  I have both libs building, and I'm going to need to make the actual app here in a second so it would help to know what you list of things is that I need to dump.
[02:26] sean_nothings: Well, basically we want to make vorbis dump out the things it's decoding as it decodes each frame. So there are two things: one the huffman decoding / VQ stuff at the low level, and at the high level, the spectral data it's decoding
[02:27] sean_nothings: the latter is probably more important than the former, but if we can get both that's good
[02:27] sean_nothings: the spectral data is decoded in several phases and combined, so we'll want to dump what it decodes at each phase (if it matches the spec), or the cumulative value so far (if that matches the spec)
[02:28] sean_nothings: that's the overview, but I can be very specific when you want to start going into the source
[02:28] muratorigo: OK I need a minute to organize things here...
[02:29] sean_nothings: what I'd do is have your sample app have a 'FILE *decode_log' and set that up in the sample app, then just extern that in the vorbis library and fprintf to it in whichever places you can get at the data
[02:29] sean_nothings: (this is probably trivially obvious but I'm just going to state everything I can think of since I've been thinking about it for longer than you and I don't know what's obvious or not)
[02:31] muratorigo: I agree.
[02:32] muratorigo: I assume you are clean-rooming Ogg as well?  Not just Vorbis?
[02:32] sean_nothings: yeah, but that's trivial
[02:32] sean_nothings: I'll be able to look and see things going wrong, since it's all on byte boundaries
[02:32] muratorigo: OK.
[02:32] sean_nothings: the super-compressed stuff is rough because all bitstreams are valid, so if you get out of sync it's not obvious
[02:33] sean_nothings: but all the header-ish stuff should be fine
[02:34] sean_nothings: plus the final IMDCT just shuffles the data insanely so you lose all sense of what's going on--a lot like the thing we were talking about cryptography functions shuffling data so every input affects every output
[02:34] sean_nothings: ok, I guess maybe 'one hour' was an underestimate if it's taking us 25 minutes to get to the sample app built
[02:35] sean_nothings: (not a complaint, an apology)
[02:37] muratorigo: Oh I think it's going to be way more than an hour :)
[02:37] sean_nothings: I guess I was figuring _I_ could do it in an hour
[02:37] muratorigo: But no matter how long it takes, it's going to be a hell of lot less than it would've taken me to do it myself!
[02:37] sean_nothings: heh
[02:37] muratorigo: Anyway I have everything building here in a custom project.
[02:38] muratorigo: I will now get those examples files playing and verify that they sound proper.
[02:38] sean_nothings: the first one is just a click
[02:41] muratorigo: The real maddening thing so far is that since we are cleanrooming, I cannot send you lots of snide comments I want to make about their source and sample apps :)
[02:41] sean_nothings: heh
[02:41] muratorigo: Anyway I'm almost ready, sorry for the delay.  You have to remember that this is the first time I've built a project in MSVC's IDE in... uh.... five years??
[02:42] sean_nothings: oh, I didn't realize you didn't use the ide, ok
[02:42] sean_nothings: anyway, no apologies needed for the time taken at all
[02:42] sean_nothings: this is going to save me a ton of frustration, I'm sure
[02:43] sean_nothings: (I could have waited until the first time I actually hit one of these bugs to ask for it, so I can only speculate, but from reading the spec it's pretty clear the peril)
[02:48] muratorigo: OK I think we're good to go.
[02:49] muratorigo: I have a program now that will read Ogg Vorbis files and write out the raw PCM data, which I've loaded back into GoldWave and verified that it is equivalent.
[02:49] sean_nothings: cool
[02:49] muratorigo: What should we head for first?
[02:49] sean_nothings: let's do the top-level structure
[02:50] sean_nothings: er, the top-level sample/spectral stuff
[02:50] muratorigo: Are we just getting these dumps and then you're going to go work on it, or are you actually going to be writing the code to match the dumps as we go?
[02:50] sean_nothings: we're just getting the dumps
[02:50] muratorigo: Got it.
[02:50] muratorigo: OK, sample spectral stuff.
[02:50] sean_nothings: (some specifications sometimes have actual test streams like this and such)
[02:50] sean_nothings: ok
[02:50] sean_nothings: so, the basic decode pipeline is this:
[02:51] sean_nothings: you have a frame of N samples, this comes from "blocksize_0" or "blocksize_1"
[02:51] sean_nothings: (when I use symbolic names, these are the names in the spec)
[02:51] sean_nothings: and there will be audio_channels many of them per frame
[02:51] sean_nothings: however, the spectral data, if I understand it correctly, is only half as much data
[02:52] sean_nothings: still, the odds are that there are floating point 'sample buffers', and it decodes the spectral data into them, then runs the IMDCT on them 'in place' to generate the final output, even if they're not the same size
[02:52] sean_nothings: anyway, I'll describe the overall pipeline, and if we're lucky, there's a function in there that calls each step of the overall pipeline, and after each step we just dump out the current sample buffers
[02:52] muratorigo: Roger.
[02:53] sean_nothings: should you want to reference something without asking me, the relevant bit of spec is here: (Link: http://xiph.org/vorbis/doc/Vorbis_I_spec.html#id2545699)http://xiph.org/vorbis/doc/Vorbis_I_spec.html#id2545699
[02:53] muratorigo: Got it.
[02:53] sean_nothings: so the central thing is that we do the following: decode a 'floor' vector, decode a 'resiude' vector, elementwise-multiply them together, IMDCT, and then apply a window
[02:54] sean_nothings: I don't care about the window, so what I'd like to capture is the floor vector, the residue vector, the multiplied one (which the spec calls a 'dot product' but it's not), and the post IMDCT data
[02:54] muratorigo: OK.
[02:54] sean_nothings: since I don't know what the size of any of these things are, assuming they're all flaots, you should just use %g fprintf format to print them
[02:55] muratorigo: OK so I'm looking for a floor vector dump, a residue vector dump, a floor(OP)residue vector dump, and an IMDCT'd vector dump.
[02:55] sean_nothings: s/size/magnitude/
[02:55] sean_nothings: yeah. and if some of those don't exist because they've combined steps, that's fine
[02:55] sean_nothings: but the trick to doing this fast is to find the 'top-level' routine that invokes each of those
[02:55] sean_nothings: which you can do by reading the source, or by finding one of those steps and trapping in the debugger and looking up the stack
[02:56] muratorigo: I already know where all of these things are.
[02:56] sean_nothings: at the very _start_ of the packet decode it decodes a 'mode number' and a 'mapping'
[02:56] sean_nothings: dumping those might be good too
[02:57] sean_nothings: do you know where all those are in detail (e.g. the specific pieces of data we need), or just in general terms?
[02:57] sean_nothings: because, like, it's still probably easier to find a high-level caller and just repeatedly dump the same buffer there, if that's possible
[02:58] muratorigo: I will look for that first but I don't know that I am going to find that.
[02:58] sean_nothings: ok
[02:59] sean_nothings: yeah, it's possible each one just chains to the next one, or that it's all a horrible data-driven thing where they're totally abstract 'steps' in a 'do N steps' and each one's a function pointer call
[02:59] sean_nothings: the former is plausible but the latter is gone too far
[03:00] sean_nothings: although there are multiple types of residues and floors so those could be function-pointer driven
[03:00] sean_nothings: (but that's different from a loop over function pointers, which would be horrific)
[03:00] sean_nothings: (I'm providing you with material to snark about, even though you can't snark back)
[03:01] sean_nothings: If I'm guessing right, the floor decode has to happen first (because of the stream), but the residue decode needs to _accumulate_ in multiple passes, so the residue decode can't decode directly on top of the floor decode. So there actually has to be two buffers for those
[03:01] sean_nothings: vs. what I was saying about there just being one buffer being reused at each step
[03:02] sean_nothings: but still I'd bet 80% of them reuse the same buffer
[03:02] sean_nothings: oh yeah, there's an inverse coupling step I left out
[03:02] sean_nothings: after residue decode, the output of that is inverse coupled, _then_ the floor is element-wise multiplied
[03:03] muratorigo: I suppose I should also dump with type of floor/residue is being used, for coverage purposes, since if none of the ogg files actually use a type, then we don't have a dump/test for it.
[03:03] sean_nothings: so catching the result of inverse coupling would be good too, if it's convenient. actually I geuss the residue * floor function is a good place to start
[03:04] sean_nothings: well, Jeff doesn't care about the floor 0 and res 0 cases because they're old
[03:04] sean_nothings: I may come back later and address them, but I don't care if it turns out we don't get coverage of them that much
[03:04] sean_nothings: but if it's handy, sure
[03:07] sean_nothings: If you're actually to the point of printing anything to the log, let me know, because I have some formatting suggestions
[03:07] muratorigo: Yeah I'm just doing some now.  Printing out the blockside, bitrate, etc.
[03:07] muratorigo: (side -> size)
[03:07] sean_nothings: don't worry about that stuff too much, that's easy to decode in the header (byte boundaries again)
[03:08] sean_nothings: the 'mode' at the start of the packet encapsulates everything, so just that is enough
[03:08] sean_nothings: well, I won't begrudge the other stuff, but I'm trying to stay laser-focused on the places I expect will be hard
[03:08] muratorigo: I'm waiting for these formatting suggestions :)
[03:08] sean_nothings: (based on my experience with jpeg & png)
[03:08] sean_nothings: oh yeah
[03:09] sean_nothings: well, this about dumping big sample/spectral buffers, not individual random things
[03:09] muratorigo: OK
[03:10] sean_nothings: which is: I'd just go ahead and print them all on one "line", but print each channel (or channel-like thing) on a separate line
[03:10] sean_nothings: that makes diffing harder but I think the readability will help
[03:11] sean_nothings: (I'd rather be able to see 1000 numbers from the log at once then only see 40)
[03:14] muratorigo: So you just want 1000 floats on one line if that's how big the channel is?
[03:14] sean_nothings: if that's how big the blocksize or whatever is, yeah
[03:16] muratorigo: Alrighty.
[03:26] sean_nothings: are you still looking for things, or just merrily instrumenting?
[03:27] muratorigo: Merrily instrumenting.
[03:27] sean_nothings: Also, I guess the reason not to log things I didn't ask about is because they might not be defined by the spec (at least as part of that stage of the pipeline)
[03:29] muratorigo: I'm only logging the things you asked for, plus stuff at the beginning that comes from the file so you can verify that you're actually thinking about the stream the same way they are.  I realize you can do that stuff without a log, but it seems like it would be a good idea to have the log for what it thought the file values were too in a few places.
[03:29] sean_nothings: ok, sure
[03:32] muratorigo: (I'm assuming that you have stuff to work on there so you're not sitting around waiting for me :)
[03:32] sean_nothings: yeah
[03:32] sean_nothings: I'm just doing the basic ogg stuff first
[03:50] sean_nothings: Where do we stand? I don't want you to put too much time into this
[04:02] muratorigo: We're all good.
[04:02] muratorigo: Don't worry about me at all :)
[04:02] muratorigo: It saves me time even if it takes a few days!
[04:02] muratorigo: But I have everything isolated
[04:03] muratorigo: I'm just trying to make sure I'm dumping the right things.
[04:03] muratorigo: The code is not so clean.
[04:03] sean_nothings: hmm, I'm not sure how you can be sure
[04:03] muratorigo: I'm watching it in the debugger.
[04:03] sean_nothings: right, but how do you know if it's what you think it is even then?
[04:03] sean_nothings: oh I see
[04:03] sean_nothings: that what you print is what's being used in an appropriate way, or whatever
[04:04] muratorigo: Yeah.  I mean I can't be positive, but I can be pretty sure.
[04:04] muratorigo: And making sure the lengths of buffers are what I think they are, et.
[04:05] sean_nothings: I've got the basic vorbis header parsing (since you kind of blow off ogg parsing to do that anyway)
[04:13] muratorigo: You do want %g for all these right?
[04:13] sean_nothings: yeah
[04:33] muratorigo: How many types of residual handling do you believe there to be?
[04:34] sean_nothings: there is a single 'type', and 3 'modes'
[04:34] sean_nothings: according to the spec, mode 2 can be defined by wrapping mode 1, though it need not be
[04:34] sean_nothings: and indeed mode 0 and mode 1 are pretty similar, they just interleave some stuff differently
[04:34] muratorigo: OK good.
[04:35] sean_nothings: but this was why I was proposing the high-level thing; if you just dump the residue at the high-level, you don't have to do it in 3 (or however many) places
[04:40] muratorigo: For reasons that I will not go into due to the cleanrooming, it is not possible to do what you are describing, but it is also not a problem, I'm pretty familiar with the code and it's no big deal.
[04:40] sean_nothings: ok
[04:40] muratorigo: The actual reason I asked was because I am doing coverage testing and I thought we hadn't any tests that used one of the types, but I just finished and it turns out the test5 file does hit the one case that was missing.
[04:40] muratorigo: Anyway I'm pretty much done here I'm just cleaning things up.
[04:41] sean_nothings: ah, test5 is probably an old file, so it might be exercising the old formats that I'm not going to do anyway
[04:41] sean_nothings: well, or not for RAD, anyway
[04:42] sean_nothings: also, this was only part 1 of the two parts of the system I was interested in logging. do you want to keep going? the other one probably isn't as important
[04:43] muratorigo: I need more time on this one first.
[04:43] muratorigo: Can you explain to me what is happening with this coupling step as far as you know?
[04:44] muratorigo: (the floor and dct parts I am pretty sure are correct now, but the residual and coupling parts are hairier)
[04:44] sean_nothings: there is a spectral vector (blocksize/2) for the residual, and one for the floor, and they are multiplied together
[04:44] sean_nothings: elementwise
[04:44] sean_nothings: the output of that is the input to the IMDCT
[04:44] sean_nothings: so if you output the input to the IMDCT, you've got the post-coupled data
[04:45] sean_nothings: and if you find that multiplication, you can print out the residual there
[04:45] sean_nothings: let me know if you need more
[04:46] muratorigo: Hmm.  Yes I definitely print out the result of that step, but this is a different thing I am looking at here.
[04:46] sean_nothings: as I mentioned before, there's a conflict because the floor comes first in the file, but the residual can't be immediately multiplied against the floor as you go
[04:46] sean_nothings: so it's possible there's trickery for that
[04:46] sean_nothings: the result of that step?
[04:47] sean_nothings: oh wait, coupling
[04:47] sean_nothings: sorry
[04:47] muratorigo: I am definitely printing out the result of the floor/residual multiplication.
[04:47] sean_nothings: the thing above was... right
[04:47] sean_nothings: ok
[04:47] sean_nothings: coupling
[04:47] sean_nothings: so, you have M channels of N/2 residual pieces of data
[04:47] sean_nothings: except those aren't necessarily the "real" final channels
[04:47] sean_nothings: because it does stereo coupling sorts of things
[04:48] muratorigo: Tell me again, specifically, without leaving any little thing out at all, what every step is that you think should be there.
[04:48] sean_nothings: so there's... submaps? let me look it up
[04:48] sean_nothings: be there for coupling, or overall?
[04:48] muratorigo: the whole thing.
[04:48] sean_nothings: ok, from the spec:
[04:49] sean_nothings: 2. read ilog([vorbis_mode_count]-1) bits [mode_number]
[04:49] sean_nothings: the first thing read is 1 bit, which is basically discarded, then there's the above mode number
[04:49] sean_nothings: the spec talks about the windowing function at that point, but it's not really used until later (and I said I don't care much about it)
[04:50] sean_nothings: ok, then the floor is decoded, then the residue, then the residue is "inverse coupled" (really it's coupled, since the encoder decouples, presumably)
[04:50] sean_nothings: details on inverse coupling in the spec are here: (Link: http://xiph.org/vorbis/doc/Vorbis_I_spec.html#id2545699)http://xiph.org/vorbis/doc/Vorbis_I_spec.html#id2545699
[04:51] sean_nothings: scroll down to "4.3.5 inverse coupling"
[04:51] sean_nothings: it's pretty trivial
[04:51] muratorigo: OK that was what I was looking for.
[04:51] sean_nothings: I don't need intermediate steps from that
[04:51] muratorigo: I wanted to make sure that wasn't something unspec'd, because I was looking at it and you hadn't described that.
[04:51] sean_nothings: just 'the residue vector for all channels' before it, and after it
[04:51] sean_nothings: ah ok
[04:52] sean_nothings: although if it's overlapped with the floor*residual, it may not be available directly, in which case ignore it
[04:52] muratorigo: I think we're pretty good here, then.
[04:52] sean_nothings: cool
[04:53] muratorigo: There's a slight problem which is that the 8 meg OGG you sent dumps to like a gig of ASCII.
[04:53] sean_nothings: do you have a program that can truncate it?
[04:53] sean_nothings: I'm sure all I need is like one frame (that's not all 0s), so 4-10 frames will be fine
[04:53] muratorigo: Yeah it's called my program :)  I can easily just stop it after the file gets to a certain size.
[04:54] sean_nothings: this stuff isn't if-tastic, so if there are bugs they should always show
[04:54] sean_nothings: (assuming basic coverage)
[04:54] muratorigo: Alrighty.
[04:54] muratorigo: I'll send you a test log in a second here so you can make formatting changes and I can put them in.
[04:54] sean_nothings: ok
[05:15] muratorigo: I sent you an e-mail with a log.
[05:16] muratorigo: I am working on the header stuff now (books, floors, maps, etc.)
[05:16] sean_nothings: -4.31602e+008 looks suspiciously wrong
[05:16] sean_nothings: I wouldn't spend any time on the header stuff
[05:16] muratorigo: Where are you looking.
[05:16] sean_nothings: too little benefit for the risk of exposing something not in the spec
[05:16] sean_nothings: the first residual
[05:17] sean_nothings: hmm, you don't print out the blocksize there, let me check the header
[05:17] sean_nothings: ok
[05:17] muratorigo: Sorry that is just because I'm printing out longer than I should for that one.
[05:17] muratorigo: I will fix that.
[05:17] sean_nothings: it's mode 0, mode 0 is block 0, and block 0 is 256, so it should be half that... right
[05:17] muratorigo: I do the same thing on the pre-MDCT - only the first 128 values are valid.
[05:18] muratorigo: I can go trunc those for you though.
[05:18] sean_nothings: yeah, I don't want to see the garbage
[05:18] sean_nothings: everything before MDCT should be half the blocksize
[05:19] sean_nothings: what's pre-reconstruction post-reconstruction?
[05:19] sean_nothings: and why is it 19 values? I guess that's an internal step of the floor? get rid of that
[05:20] muratorigo: OK I will get rid of the floor values.
[05:20] sean_nothings: there's probably something useful there, but I'm suspiciously I'm seeing internals
[05:21] sean_nothings: I guess pre-db/post-db is the full floor data so that should be plenty
[05:21] sean_nothings: this looks totally awesome otherwise
[05:22] muratorigo: I'll #if 0 out the 19 floor part, because I'm pretty certain that's it's spec'd stuff, so when you get to that part if you find "oh here's some floor stuff we don't have dumped", I can put it back in because that's probably what it is.
[05:22] sean_nothings: yeah, there's something where that length is reasonable, I'm just suspicious of _two_ versions of it
[05:22] sean_nothings: and I think if I have the other thing I wanted it will be redundant anyway
[05:23] muratorigo: OK.
[05:23] sean_nothings: of if you really want to do floor stuff, I have a better thing to dump that's clearly spec-defined
[05:23] sean_nothings: or if
[05:25] sean_nothings: but really, we can just stop here, because three hours is not ONE HOUR
[05:28] muratorigo: Actually I think I'm going to hit the sack now, but I would be happy to do the rest of the dumping tomorrow.  Why don't we chat again tomorrow?
[05:28] muratorigo: (ps I sent you another version of the dump with the formatting changes we just discussed)
[05:28] sean_nothings: why don't you dump all 5 files with this version (tomorrow or not), and hold off on doing anything more until I see if I need it
[05:28] sean_nothings: er, tomorrow or now, no rush
[05:29] muratorigo: What is the thing you want to dump, btw?
[05:29] sean_nothings: Ok, this looks awesome, although it's mono so I'm not sure how multiple channels are going to come out
[05:29] sean_nothings: since you're printing the label on the same line as the numbers
[05:30] sean_nothings: IF the code hasn't inlined the huffman decode, then log the result of every huffman decode (just in the low-level function)... what the spec calls a codebook decode in a scalar context
[05:30] sean_nothings: since it would be difficult for this to be done out-of-order, dumping it at the decode step should be well-defined
[05:31] sean_nothings: but I suspect I'll be able to catch this stuff directly
[05:32] sean_nothings: the floating data would be harder to work backwards to figure out what's going wrong, without info from each step
[05:32] sean_nothings: wrong huffman data will go wrong more obviously, I think
[05:34] muratorigo: Hmm.
[05:34] sean_nothings: well, don't worry about it now!
[05:34] sean_nothings: for instance, wrong huffman interpretation will mean these numbers in this log will go wrong
[05:35] sean_nothings: so if I have bugs I'll know which step it happens in, regardless
[05:35] sean_nothings: so seriously, tomorrow, address the channel issue (if there is any; I just need to make sure it's clear in the file), and generate the logs, and we'll see
[05:35] muratorigo: So what exactly would this log look like, of the huffman decode?
[05:36] sean_nothings: printf("[%d]", value_corresponding_to_huffman_code_that_is_about_to_be_returned_by_this_leafish_function)
[05:37] sean_nothings: the codebook scalars are integers. they have corresponding huffman lengths, and corresponding huffman codes are generated. the stream is huffman decoded and a codebook scalar entry computed
[05:38] sean_nothings: if we were really lucky, the codebook scalar decode function would pass in the codebook by _number_ and you could print that too, but it probably is a function that takes a pointer to a codebook or whatever
[05:39] sean_nothings: but for speed reasons they might just inline decode the huffman stream in the various different places, instead of calling a common function, in which case it's definitely not worth worrying about
[05:39] muratorigo: So you would print ("[%d, %d]", Codebook, Result)?
[05:39] sean_nothings: yeah
[05:39] muratorigo: Without getting into specifics, let it suffice to say that I don't think there is much in this codebase that was done for "speed reasons".
[05:39] sean_nothings: aw
[05:40] sean_nothings: anyway, seriously, don't spend more time on it
[05:40] sean_nothings: I realize you figure you can do it now (well, tomorrow) to save having me come bug you for more later, but I really doubt I'll need it
[05:57] sean_nothings: or you can disappear for 20 minutes because you decided to do it anyway, I guess
[06:04] muratorigo: Hmm.
[06:05] muratorigo: Well, I have no idea if this is actually printing out the right thing, but...
[06:05] muratorigo: ... I sent you a dump of the huffman thing.  Does that look remotely plausible
[06:06] muratorigo: They do not pass the codebook by index, so I had to add some wiring, but I think it's pretty reliable.  In fact I'm much more confident in the codebook index than I am that the value is the actual value you want it to be.
[06:07] sean_nothings: they all end in d... did you say %dd?
[06:08] sean_nothings: well, either the value is the true value, or it's the huffman-coding of it, and either one will work
[06:08] muratorigo: It's the time of night when that is certainly possible...
[06:08] sean_nothings: that's totally fine. if you log them both to the same file at the same time, that will be fine
[06:08] sean_nothings: thanks a ton
[06:08] muratorigo: Yeah there was an extra d.
[06:09] muratorigo: Wait, you want these values to just appear interspersed in the other file?
[06:09] muratorigo: They're going to be everywhere.
[06:09] sean_nothings: sure
[06:09] sean_nothings: that's fine
[06:14] muratorigo: OK I sent you a file with what that looks like... is that what you want?
[06:16] sean_nothings: hmm, they're actually all only printing in one place?
[06:16] sean_nothings: well, whatever
[06:16] sean_nothings: yeah, it's great. ship it
[06:16] muratorigo: OK.
[06:17] sean_nothings: do the others whenever, this one alone is enough to get me started
[06:17] sean_nothings: since I assume this is test1
[06:22] muratorigo: I am just waiting for 7zip to finish zipping up the complete set.
[06:22] sean_nothings: ah cool
[06:22] muratorigo: The longer ones are truncated.
[06:22] sean_nothings: did you trunc... ok
[06:23] muratorigo: Yes.  When it gets to the 5000th float array, it bails.
[06:31] muratorigo: (Link: http://funkytroll.com/temp/tests.7z)http://funkytroll.com/temp/tests.7z
[06:33] sean_nothings: cool. thanks a bazillion
[06:33] muratorigo: Thanks for writing the code, dude!
[06:33] muratorigo: Do those files look OK?
[06:33] sean_nothings: still dling
[06:36] sean_nothings: looks good!
[06:36] sean_nothings: thanks!
[06:37] muratorigo: Cool beans...
[06:37] muratorigo: I'll save all the project stuff so if we need to do more dumps as you progress, we can.
[06:37] muratorigo: Good luck!
[06:37] sean_nothings: right
[06:37] sean_nothings: ok!
[06:37] muratorigo: 'night.
[06:37] sean_nothings: goodnight
[06:38] sean_nothings: say goodbye to the readers
[06:41] sean_nothings: bye readers!
Session Close (muratorigo): Mon Apr 02 06:41:37 2007