Editor support for Chrysler 2d tables

[vprtech]

Since it's looks like we are getting a little closer to having support for SBECII calibrations, I thought I might bang on some doors to find someone, who already has a decent bin editing software, to add support for the 2D table format used on these controllers. One of the persons I have contacted about this is Mark Mansur who is the developer of Tuner Pro , he seems open to the idea of adding support , but wants some more details as to exactly how these tables work, and what are the different variations of these table, ie signed/unsigned , 8 bit and 16 bit, etc . I believe that is the only table or variable format that is significantly different than what would be found on other vehicle types such as Ford or GM, and this might be a good opportunity to have a editor that does not have any bugs displaying the 2D format, as I believe one of the existing editors has a problem with the last breakpoint on certain tables. My question is, what sort of information should I forwarded to the software developer that would give him all of details as to how the 2D tables work, and how they should be integrated into the software. I have already sent an example of one of these tables, both in hex format and graphically. Hopefully I can get Rob to chime in here, but any suggestions or input would be appreciated. Also, if someone can think of any other Chrysler format , table , or constant , that is unique to Chrysler, please let me know, as I will try to get this supported as well.

[risen]

I can give you a breif english write up of how the 8 bit tables and 16 bit fueling tables are laid out for LM/SMEC based cals. I believe that the sbec's are the same for the most part (minus the 3d stuff). I think knowing things like how the values before and after the first and last points are returned and that there's slope bytes stuck in the middle of the values for the table are invaluable bits of info.

I can give you a brief english write up of the 8 bit and 16 bit table formats if you'd like. I also have some perl code that will read a cal and pull the x and y from the tables if that would be helpful, too.

[vprtech]

I can give you a breif english write up of how the 8 bit tables and 16 bit fueling tables are laid out for LM/SMEC based cals. I believe that the sbec's are the same for the most part (minus the 3d stuff). I think knowing things like how the values before and after the first and last points are returned and that there's slope bytes stuck in the middle of the values for the table are invaluable bits of info.

I can give you a brief english write up of the 8 bit and 16 bit table formats if you'd like. I also have some perl code that will read a cal and pull the x and y from the tables if that would be helpful, too.

Great Risen, I believe that info would be very helpful. I'm fairly certain that Tuner Pro will be limited in that there will be no table scanning, like in DCAL, the definitions will have to be provided, although I'm not sure if the table length and signed/unsigned status will have to be set in the definition, or if Mark can make it automatically look at the header and identify the table signed/unsigned status and length like DCAL does.

[risen]

Great Risen, I believe that info would be very helpful. I'm fairly certain that Tuner Pro will be limited in that there will be no table scanning, like in DCAL, the definitions will have to be provided, although I'm not sure if the table length and signed/unsigned status will have to be set in the definition, or if Mark can make it automatically look at the header and identify the table signed/unsigned status and length like DCAL does.

I've looked at tunerpro's definition format and it should be a trivial task (famous last words) to convert our .tbl or .calx files into tunerpro's format. I think the signed/unsigned and type of table can all be specified in the definition file. I I have Mon/Tue off from work, so I should be able to have it done by then.

[risen]

Just out of curiosity, is there a reason that you didn't want to use dcal or chem2? Don't get me wrong, I'm all for having additional choices, but those other 2 programs work fine. I normally do everything but the bit switches in chem2 and it's always been good to me. And I've had the same pistons in my CSX @ 20psi of boost for around 4 years now :) . Also, there's at least two ASM editors in the works.

I'm still looking at the slope calculation code from chem2 to make sure that I understand the slope calculations completely. There's an 'and' in there and one or 2 other things that I don't quite understand, but what's below should be a good start. Hopefully if I'm not getting something Rob or someone else will chime in and straighten my mistake(s) out.

8 bit 2d tables:
1st byte in table is number of entries in table (n)
Thereafter each entry in the table is 4 bytes long (to n - 1) has the following format
1st byte in entry - X value - Comparison to table's input
2nd byte in entry -Y value - Return from compare
3rd byte in entry - Slope1 - High byte, whole number of slope
4th byte in entry - Slope2 - Fractional part of slope

Each of the entries is the same until the last entry, final entry in table has following format:
X value - Compare to table's input
Y value - Return

Prior to the first point and after the final point in the table the slopes are flat. This means that any value less than the first X value in the table will return the Y value of the first point in the table. Any X value after the last point in the table will return the Y value of the last point in the table.

The 16 bit tables are almost the same:

The table begins with the entries, there is no count as the first entry. The entries are all of the following format:
1st byte - X Value for compare
2nd & 3rd bytes (16 bit value) - Y Value for output
4th byte - Slope1 - High byte, whole number from slope
5th byte - Slope2 - Low byte, fractional part of slope

The entries in the fueling tables go on until there is an 0xff after the last entry in the table. That's how they all end from what I can tell. After the last point in the 16 bit tables the slope is extended to the maximum input of the table (0xff). I believe the first entries are all at X=0x00, so I don't think there's a special case to worry about for x values less than the first x point in the table.

Constants are essentially the same as any other ECU. The older stuff is all 8 bit, some signed, some unsigned but I think that can be taken care of by the xdf. I don't know when chrysler went to an adc resolution > 8 bit.

There's still all sorts of corner case little stuff (like the sqrt tables and bit flags) but I think if he gets this much supported we'll be at the 90% of the 90/10 rule ( the last 10% being the hardest).

[ShelGame]

I think you nailed it pretty good.

Another thing about the point/slope table storage is that the slope doesn't necessarily have to end up at the next point. In fact, there are a couple of cases where the table appears to intentionally use this effect. CHeM2 'fixes' these by recalculating the slope if you edit that particular table (Geoff assumed these were 'errors' by Chrysler - I think Chrysler just used the discontinuity to an advantage in some cases). Essentially, this gives you the ability to have 2 data points for the price of 1 - in rare cases.

I always thought there should be a way to adjust the slope separately from the point data. But, none of our current editors can do that. And, again, it's a pretty rare case.

[vprtech]

Just out of curiosity, is there a reason that you didn't want to use dcal or chem2? Don't get me wrong, I'm all for having additional choices, but those other 2 programs work fine. I normally do everything but the bit switches in chem2 and it's always been good to me. And I've had the same pistons in my CSX @ 20psi of boost for around 4 years now :) . Also, there's at least two ASM editors in the works.

I'm still looking at the slope calculation code from chem2 to make sure that I understand the slope calculations completely. There's an 'and' in there and one or 2 other things that I don't quite understand, but what's below should be a good start. Hopefully if I'm not getting something Rob or someone else will chime in and straighten my mistake(s) out.

8 bit 2d tables:
1st byte in table is number of entries in table (n)
Thereafter each entry in the table is 4 bytes long (to n - 1) has the following format
1st byte in entry - X value - Comparison to table's input
2nd byte in entry -Y value - Return from compare
3rd byte in entry - Slope1 - High byte, whole number of slope
4th byte in entry - Slope2 - Fractional part of slope

Each of the entries is the same until the last entry, final entry in table has following format:
X value - Compare to table's input
Y value - Return

Prior to the first point and after the final point in the table the slopes are flat. This means that any value less than the first X value in the table will return the Y value of the first point in the table. Any X value after the last point in the table will return the Y value of the last point in the table.

The 16 bit tables are almost the same:

The table begins with the entries, there is no count as the first entry. The entries are all of the following format:
1st byte - X Value for compare
2nd & 3rd bytes (16 bit value) - Y Value for output
4th byte - Slope1 - High byte, whole number from slope
5th byte - Slope2 - Low byte, fractional part of slope

The entries in the fueling tables go on until there is an 0xff after the last entry in the table. That's how they all end from what I can tell. After the last point in the 16 bit tables the slope is extended to the maximum input of the table (0xff). I believe the first entries are all at X=0x00, so I don't think there's a special case to worry about for x values less than the first x point in the table.

Constants are essentially the same as any other ECU. The older stuff is all 8 bit, some signed, some unsigned but I think that can be taken care of by the xdf. I don't know when chrysler went to an adc resolution > 8 bit.

There's still all sorts of corner case little stuff (like the sqrt tables and bit flags) but I think if he gets this much supported we'll be at the 90% of the 90/10 rule ( the last 10% being the hardest).


This is very helpful, I will forward this to Mark.

There are a few reasons why I want to use TunerPro as opposed to CHEM or DCAL , some of these reasons are because I'm trying to tune SBEC 2 ecus .

- Supports real time emulator use, when you change calibration data, it updates immediately .

- You can edit 3D tables graphically, and TPro links the normalizers to the table X and Y axis, I don't believe DCAL or CHEM can do this.

- If someone can get the SCI data logging to work with TunerPro, you will be able to view the PIDs in the TunerPro workspace, through a strip chart or numerically, you can also datalog.

- Again, if we can get the logging to work, you can link the table normalizers to the appropriate PIDs, and get linked map tracing.

- I believe, when using a Ostrich emulator, you can get hardware based map tracing, useful when you want to know at a glance what table the ecu is accessing. I use this with my Racelogic emulator, it might not sound very useful, but it has been a big help to me several times.

I did not know that there were other editors in the works, could you expand on this ? Do you know if they support emulation and 3D tables used in the later T3/SBEC 2 calibrations ?

[risen]

I think you nailed it pretty good.

Another thing about the point/slope table storage is that the slope doesn't necessarily have to end up at the next point. In fact, there are a couple of cases where the table appears to intentionally use this effect. CHeM2 'fixes' these by recalculating the slope if you edit that particular table (Geoff assumed these were 'errors' by Chrysler - I think Chrysler just used the discontinuity to an advantage in some cases). Essentially, this gives you the ability to have 2 data points for the price of 1 - in rare cases.

I always thought there should be a way to adjust the slope separately from the point data. But, none of our current editors can do that. And, again, it's a pretty rare case.

Hmm, interesting. But now I'm wondering about when the ecu is running how does it know to look for the slope in a different place? Is there some sort of flag? Just another instance of ChryCo's engineers being too clever for their own good :)



This is very helpful, I will forward this to Mark.

There are a few reasons why I want to use TunerPro as opposed to CHEM or DCAL , some of these reasons are because I'm trying to tune SBEC 2 ecus .

- Supports real time emulator use, when you change calibration data, it updates immediately .

- You can edit 3D tables graphically, and TPro links the normalizers to the table X and Y axis, I don't believe DCAL or CHEM can do this.


There was another thread about the 3d tables ( in which I had suggested using tunerpro for 3d tables) which you should have a look at. I forget the exact details, but the input and output from the table is non-linear. So, while graphing and editing the table should be a snap, understanding what goes in and comes out isn't so easy. Now, my question is, is the non-linearity of the input or output what the normalizers do? Apply a xfer function to the input/output of the tables?




- If someone can get the SCI data logging to work with TunerPro, you will be able to view the PIDs in the TunerPro workspace, through a strip chart or numerically, you can also datalog.

- Again, if we can get the logging to work, you can link the table normalizers to the appropriate PIDs, and get linked map tracing.

- I believe, when using a Ostrich emulator, you can get hardware based map tracing, useful when you want to know at a glance what table the ecu is accessing. I use this with my Racelogic emulator, it might not sound very useful, but it has been a big help to me several times.

I recently helped do a little tuning on an eec-iv with a quaterhorse and binary editor. When coupled with the proper software, hardware tracing is incredibly powerful, so I hear you there.

I'm sure we can get datalogging to work given enough will power :) .



I did not know that there were other editors in the works, could you expand on this ? Do you know if they support emulation and 3D tables used in the later T3/SBEC 2 calibrations ?

The other editors are (as far as I know) for the .asm files of the relocatable source cals. So, since you're only interested in the sbec2 stuff, I doubt they're of much interest to you. Unless they're going to support binary editing also, but I don't know what features are or aren't planned for the editors that are in the works.

[ShelGame]

Hmm, interesting. But now I'm wondering about when the ecu is running how does it know to look for the slope in a different place? Is there some sort of flag? Just another instance of ChryCo's engineers being too clever for their own good :)

Probably the latter.

It's not a matter of looking for a different slope; it's just that (since the slope is encoded along with the X,Y coords), it's possible to have a slope that does NOT lead to the next point. I'll see if I can find an example to illustrate.

But, in basic terms, think about 2 points. Point A is at X,Y = 1,1. Point B is at 3,2. So, the slope of a straight line between them would be 0.5. But, if you encoded a slope of 1, the theoretical point would be 3,3. So, in effect, 1 bit before the next point would be nearly 3,3, and then the point would be 3,2. It would look like a saw-tooth in this case.



The other editors are (as far as I know) for the .asm files of the relocatable source cals. So, since you're only interested in the sbec2 stuff, I doubt they're of much interest to you. Unless they're going to support binary editing also, but I don't know what features are or aren't planned for the editors that are in the works.


MP Tuner will edit both .asm and .bin files. And it can compare an .asm to up to 3 other .asm's or .bin's (very usefull!).

Also, he has built-in datalogging and code tracing. I think the holdup on releasing it is probably me at this point - I need to hurry up and finish T-SMEC v16 (some changes to the source were necessary).

It also has 3D support with the proper scaling of the inputs. Though, it's just a tabular display as of right now (like D-Cal). No surface plots yet.

[risen]

Probably the latter.

It's not a matter of looking for a different slope; it's just that (since the slope is encoded along with the X,Y coords), it's possible to have a slope that does NOT lead to the next point. I'll see if I can find an example to illustrate.

But, in basic terms, think about 2 points. Point A is at X,Y = 1,1. Point B is at 3,2. So, the slope of a straight line between them would be 0.5. But, if you encoded a slope of 1, the theoretical point would be 3,3. So, in effect, 1 bit before the next point would be nearly 3,3, and then the point would be 3,2. It would look like a saw-tooth in this case.


Ahh, got it. So it's not that the layout in the binary that's different. It's that the slopes and the points aren't always contiguous. I thought it would be looking in a different place for the slope, like if X and Y were 0x00 and 0x01, and the slope was at 0xF0,0xF1 . Thanks for clearing that up.



MP Tuner will edit both .asm and .bin files. And it can compare an .asm to up to 3 other .asm's or .bin's (very usefull!).

Also, he has built-in datalogging and code tracing. I think the holdup on releasing it is probably me at this point - I need to hurry up and finish T-SMEC v16 (some changes to the source were necessary).

It also has 3D support with the proper scaling of the inputs. Though, it's just a tabular display as of right now (like D-Cal). No surface plots yet.

Can I ask how the tracing is setup? Does it just spit the address of the index register out the serial port all the time, or are there hardware mods required?

[wowzer]

the scanning basically is just a logging program, similar to what u did with the logworks plugin, allowing u to display the ram locations in digital, analog, bit, or graph form as well as save the log for later playback. the tracing is just a feature to step through compiled code at your computer, displaying all the registers, tables, ram, etc. i've used it, e.g., to trace through the 2d and 3d table lookup routines. i'll probably use it to trace through the drb stuff at a later date.

[risen]

the scanning basically is just a logging program, similar to what u did with the logworks plugin, allowing u to display the ram locations in digital, analog, bit, or graph form as well as save the log for later playback. the tracing is just a feature to step through compiled code at your computer, displaying all the registers, tables, ram, etc. i've used it, e.g., to trace through the 2d and 3d table lookup routines. i'll probably use it to trace through the drb stuff at a later date.

So the tracing is almost like running the calibration under an emulator on the host PC, correct? If so that sounds really cool, I can't wait to mess with it.

[wowzer]

So the tracing is almost like running the calibration under an emulator on the host PC, correct? If so that sounds really cool, I can't wait to mess with it.

kinda. it is a fairly simple program functionality wise. i used it several years ago when i was more into the chrysler stuff. i always wanted to make a "real time" emulator but lost interest. plus i believe there are some commercial versions out there ("thrsim") fairly cheap that go way beyond what i can (or want) to do. this just helps the "assembler challenged" people like me try to track through code.

(don't hold your breath!;))

[vprtech]

To revisit this, regarding calculating the slope of each break point, would this formula for rise over run be correct ? Is there anything I should keep in mind, and would this apply to the 16 bit tables as well ?

100 * (Y2 - Y1) / (X2 - X1)

[ShelGame]

The only issue is, there is no 'end' point in our tables. There's a last slope. Our current editors use a 'phantom' last point - used only for re-calculating the last slope.

[vprtech]

The only issue is, there is no 'end' point in our tables. There's a last slope. Our current editors use a 'phantom' last point - used only for re-calculating the last slope.

Correct me if I'm wrong, but what you are saying only applies to the 16 bit bit fuel tables, correct ? The regular tables I have looked at don't have a slope value in the bin file for the last breakpoint.

[risen]

To revisit this, regarding calculating the slope of each break point, would this formula for rise over run be correct ? Is there anything I should keep in mind, and would this apply to the 16 bit tables as well ?

100 * (Y2 - Y1) / (X2 - X1)

I believe the slope is just the standard (y2-y1)/(x2-x1) . I don't think that it's rescaled at all. It's just split into whole and fractional parts across the 2 bytes.

[ShelGame]

Correct me if I'm wrong, but what you are saying only applies to the 16 bit bit fuel tables, correct ? The regular tables I have looked at don't have a slope value in the bin file for the last breakpoint.

Actually, yes, you're right. The last slope is assumed to be zero, if the last point is NOT at x=FF.

[vprtech]

Ok, besides me being a little challenged with the hex math, I think I understand how to calculate the 8 bit tables. I am however, not clear about how the last slope value for the sixteen bit tables and the ghost break point. Specifically , how does it calculate the ghost breakpoint and the slope between the second to last and the ghost breakpoint. Is it assumed that the ghost breakpoint is at a specific max x value ?

[ShelGame]

The ECU does not use the ghost breakpoint. It doesn't even calculate it. It uses the last point and the last slope to calculate the Y (output) from the X input.

The editor is all that uses the 'ghost' point, and then it's only used for adjusting the last slope.

[wowzer]

here is how i calc the ghost point (Xb, Yb) using the last "real" point (Xl, Yl) and its slope (Sl):

Xb = 255
Yb = Yl + (Xb - Xl) * Sl / 256