Data Reduction... Further Questions
in [Design]
|
From: Jim Thompson on 13 Aug 2010 14:17 I have a data, voltage-versus-time, in 4 columns... Time DataVersion1 DataVersion2 DataVersion3 Time is in 100ps steps, over a total span of 100ns, thus 1000 data points in each data column. The end-use software, IBIS, can only handle 100 data points per column. Snag #1: Data in each column doesn't quite match data in other columns... process variations, temperature, voltage, etc. Snag #2: Time column must be the _same_ for all three data columns. Any ideas on how to process, such that enough time points can be eliminated to reduce total time points to 100. Some kind of simultaneous slope/delta analysis? Maybe some way in Excel? Thanks! Fun problem, isn't it ?:-) ...Jim Thompson -- | James E.Thompson, CTO | mens | | Analog Innovations, Inc. | et | | Analog/Mixed-Signal ASIC's and Discrete Systems | manus | | Phoenix, Arizona 85048 Skype: Contacts Only | | | Voice:(480)460-2350 Fax: Available upon request | Brass Rat | | E-mail Icon at http://www.analog-innovations.com | 1962 | Spice is like a sports car... Performance only as good as the person behind the wheel.
From: Tim Wescott on 13 Aug 2010 14:39 On 08/13/2010 11:17 AM, Jim Thompson wrote: > I have a data, voltage-versus-time, in 4 columns... > > Time DataVersion1 DataVersion2 DataVersion3 > > Time is in 100ps steps, over a total span of 100ns, thus 1000 data > points in each data column. > > The end-use software, IBIS, can only handle 100 data points per > column. > > Snag #1: Data in each column doesn't quite match data in other > columns... process variations, temperature, voltage, etc. > > Snag #2: Time column must be the _same_ for all three data columns. > > Any ideas on how to process, such that enough time points can be > eliminated to reduce total time points to 100. > > Some kind of simultaneous slope/delta analysis? Maybe some way in > Excel? > > Thanks! > > Fun problem, isn't it ?:-) What information do you want to keep, what information can you discard? If each and every voltage point in there is unique and carries valuable information, then you're screwed. If you have some prior knowledge about the voltages -- e.g. if they're bandlimited to well under 500MHz -- then there's some grist for the data-reduction mill. So, tell us more. This would be a good post for comp.dsp -- this is the sort of question we answer all the time. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" was written for you. See details at http://www.wescottdesign.com/actfes/actfes.html
From: Jim Thompson on 13 Aug 2010 14:57 On Fri, 13 Aug 2010 11:39:06 -0700, Tim Wescott <tim(a)seemywebsite.com> wrote: >On 08/13/2010 11:17 AM, Jim Thompson wrote: >> I have a data, voltage-versus-time, in 4 columns... >> >> Time DataVersion1 DataVersion2 DataVersion3 >> >> Time is in 100ps steps, over a total span of 100ns, thus 1000 data >> points in each data column. >> >> The end-use software, IBIS, can only handle 100 data points per >> column. >> >> Snag #1: Data in each column doesn't quite match data in other >> columns... process variations, temperature, voltage, etc. >> >> Snag #2: Time column must be the _same_ for all three data columns. >> >> Any ideas on how to process, such that enough time points can be >> eliminated to reduce total time points to 100. >> >> Some kind of simultaneous slope/delta analysis? Maybe some way in >> Excel? >> >> Thanks! >> >> Fun problem, isn't it ?:-) > >What information do you want to keep, what information can you discard? > >If each and every voltage point in there is unique and carries valuable >information, then you're screwed. If you have some prior knowledge >about the voltages -- e.g. if they're bandlimited to well under 500MHz >-- then there's some grist for the data-reduction mill. > >So, tell us more. > >This would be a good post for comp.dsp -- this is the sort of question >we answer all the time. It's Spice transient analysis data. I'm guessing that, at low enough or consistent slopes, you could eliminate intermediate points, since they don't really describe any "significant" change. ("Significant" is a nasty word ;-) I can do this by eye, very slowly and tediously. Maybe an Excel take-sum-of-magnitudes-of-all-three-slopes, eliminate data points below a certain slope. Sorry! I'm flailing for a solution :-) ...Jim Thompson -- | James E.Thompson, CTO | mens | | Analog Innovations, Inc. | et | | Analog/Mixed-Signal ASIC's and Discrete Systems | manus | | Phoenix, Arizona 85048 Skype: Contacts Only | | | Voice:(480)460-2350 Fax: Available upon request | Brass Rat | | E-mail Icon at http://www.analog-innovations.com | 1962 | Spice is like a sports car... Performance only as good as the person behind the wheel.
From: Tim Wescott on 13 Aug 2010 15:08 On 08/13/2010 11:57 AM, Jim Thompson wrote: > On Fri, 13 Aug 2010 11:39:06 -0700, Tim Wescott<tim(a)seemywebsite.com> > wrote: > >> On 08/13/2010 11:17 AM, Jim Thompson wrote: >>> I have a data, voltage-versus-time, in 4 columns... >>> >>> Time DataVersion1 DataVersion2 DataVersion3 >>> >>> Time is in 100ps steps, over a total span of 100ns, thus 1000 data >>> points in each data column. >>> >>> The end-use software, IBIS, can only handle 100 data points per >>> column. >>> >>> Snag #1: Data in each column doesn't quite match data in other >>> columns... process variations, temperature, voltage, etc. >>> >>> Snag #2: Time column must be the _same_ for all three data columns. >>> >>> Any ideas on how to process, such that enough time points can be >>> eliminated to reduce total time points to 100. >>> >>> Some kind of simultaneous slope/delta analysis? Maybe some way in >>> Excel? >>> >>> Thanks! >>> >>> Fun problem, isn't it ?:-) >> >> What information do you want to keep, what information can you discard? >> >> If each and every voltage point in there is unique and carries valuable >> information, then you're screwed. If you have some prior knowledge >> about the voltages -- e.g. if they're bandlimited to well under 500MHz >> -- then there's some grist for the data-reduction mill. >> >> So, tell us more. >> >> This would be a good post for comp.dsp -- this is the sort of question >> we answer all the time. > > It's Spice transient analysis data. > > I'm guessing that, at low enough or consistent slopes, you could > eliminate intermediate points, since they don't really describe any > "significant" change. > > ("Significant" is a nasty word ;-) > > I can do this by eye, very slowly and tediously. > > Maybe an Excel take-sum-of-magnitudes-of-all-three-slopes, eliminate > data points below a certain slope. > > Sorry! I'm flailing for a solution :-) So you want to actively analyze the data for information content, and only save the "important" stuff. Try plotting the data, 1st difference (derivative, slope) of the data, the second difference (2nd derivative, acceleration) of the data, and maybe even the 3rd. I suspect that what you really want to save is data that has a lot of curvature, which would be indicated by the 2nd derivative, or data that has an onset of curvature, which would be indicated by the 3rd. I'm not sure how you'd wrench Excel into discarding all but the high-curvature data -- it'd be a snap in Scilab or C or Basic or Python or any other real programming language, though. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" was written for you. See details at http://www.wescottdesign.com/actfes/actfes.html
From: Jim Thompson on 13 Aug 2010 16:13 On Fri, 13 Aug 2010 12:08:19 -0700, Tim Wescott <tim(a)seemywebsite.com> wrote: >On 08/13/2010 11:57 AM, Jim Thompson wrote: >> On Fri, 13 Aug 2010 11:39:06 -0700, Tim Wescott<tim(a)seemywebsite.com> >> wrote: >> >>> On 08/13/2010 11:17 AM, Jim Thompson wrote: >>>> I have a data, voltage-versus-time, in 4 columns... >>>> >>>> Time DataVersion1 DataVersion2 DataVersion3 >>>> >>>> Time is in 100ps steps, over a total span of 100ns, thus 1000 data >>>> points in each data column. >>>> >>>> The end-use software, IBIS, can only handle 100 data points per >>>> column. >>>> >>>> Snag #1: Data in each column doesn't quite match data in other >>>> columns... process variations, temperature, voltage, etc. >>>> >>>> Snag #2: Time column must be the _same_ for all three data columns. >>>> >>>> Any ideas on how to process, such that enough time points can be >>>> eliminated to reduce total time points to 100. >>>> >>>> Some kind of simultaneous slope/delta analysis? Maybe some way in >>>> Excel? >>>> >>>> Thanks! >>>> >>>> Fun problem, isn't it ?:-) >>> >>> What information do you want to keep, what information can you discard? >>> >>> If each and every voltage point in there is unique and carries valuable >>> information, then you're screwed. If you have some prior knowledge >>> about the voltages -- e.g. if they're bandlimited to well under 500MHz >>> -- then there's some grist for the data-reduction mill. >>> >>> So, tell us more. >>> >>> This would be a good post for comp.dsp -- this is the sort of question >>> we answer all the time. >> >> It's Spice transient analysis data. >> >> I'm guessing that, at low enough or consistent slopes, you could >> eliminate intermediate points, since they don't really describe any >> "significant" change. >> >> ("Significant" is a nasty word ;-) >> >> I can do this by eye, very slowly and tediously. >> >> Maybe an Excel take-sum-of-magnitudes-of-all-three-slopes, eliminate >> data points below a certain slope. >> >> Sorry! I'm flailing for a solution :-) > >So you want to actively analyze the data for information content, and >only save the "important" stuff. Yep. Sounds easy when you say it that way ;-) > >Try plotting the data, 1st difference (derivative, slope) of the data, >the second difference (2nd derivative, acceleration) of the data, and >maybe even the 3rd. I suspect that what you really want to save is data >that has a lot of curvature, which would be indicated by the 2nd >derivative, or data that has an onset of curvature, which would be >indicated by the 3rd. Good ideas! > >I'm not sure how you'd wrench Excel into discarding all but the >high-curvature data -- it'd be a snap in Scilab or C or Basic or Python >or any other real programming language, though. Yep, I guess I need to re-remember how to program :-( ...Jim Thompson -- | James E.Thompson, CTO | mens | | Analog Innovations, Inc. | et | | Analog/Mixed-Signal ASIC's and Discrete Systems | manus | | Phoenix, Arizona 85048 Skype: Contacts Only | | | Voice:(480)460-2350 Fax: Available upon request | Brass Rat | | E-mail Icon at http://www.analog-innovations.com | 1962 | Spice is like a sports car... Performance only as good as the person behind the wheel.
|
Next
|
Last
Pages: 1 2 Prev: Right time shockingly wrong place Next: A cheap very accurate LC meter with RS232 interface |