Scientific to Decimal
in [Visual Basic]
|
From: Webbiz on 2 Dec 2009 00:27 On Tue, 1 Dec 2009 22:53:33 -0500, "Jim Mack" <jmack(a)mdxi.nospam.com> wrote: >Webbiz wrote: >> Jim Mack wrote: >> >>> Webbiz wrote: >>> >>>> Then why are my Single values, showing in Watch as .5938 for >>>> example, showing as a Scientific Notation value after some math >>>> and assigned to another Single variable? >>>> >>>> If I send the original single value to the text box, it will show >>>> up as .5938. But do some subtraction, for example, of two of these >>>> Singles and assign to another Single, or directly to the textbox, >>>> and it displays as Scentific Notation? >>> >>> A textbox can't display a single, it can only display strings. And >>> anything you show in Watch or Debug etc is a also string. >>> >>> If you want to control how something appears on the screen (i.e. in >>> the string domain), use Format. >> >> >> Jim, I know this. > >Apparently not completely. > >Singles do not have a 'format'. They cannot change from 'regular' to >'scientific'. That is strictly a function of being displayed as a >string. If you want them to appear a certain way then you have to >format them that way. > >You are NOT seeing singles. You're seeing strings, which are the >result of formatting. Accept the default VB formatting, and you get >what VB decides. Use Format and get what you want. It's really pretty >simple. So what you're saying is that when I place my mouse over the Single variable and it displays the result of that variable during debug, it's been converted to a string without the benefit of actually being assigned to a string? Webbiz
From: Webbiz on 2 Dec 2009 00:58 On Tue, 01 Dec 2009 19:11:17 -0600, Webbiz <nospam(a)noway.com> wrote: >So if this were discovered, the string variable could be set by... > >Select Case DecCnt > Case 0 > sDataFormat = "#" > Case 1 > sDataFormat = "#.#" > Case 2 > sDataFormat = "#.##" > >...etc. > >Or is this too clumsy? I'm going to answer this one myself. I wrote this function to do the above... Public Sub SetDataFormat(ByVal Digits As Long) sDataFormat = "#" If Digits > 0 Then sDataFormat = sDataFormat & "." & Right("###########", Digits) End Sub To feed it the 'Digits', I am doing the following: If DigitsRightOfDec(CStr(Price)) > nMaxNum Then nMaxNum = DigitsRightOfDec(CStr(Price)) End If And DigitsRightOfDec is done by... Public Function DigitsRightOfDec(ByVal sNum As String) As Long If InStr(1, sNum, ".") > 0 Then DigitsRightOfDec = Len(sNum) - InStr(1, sNum, ".") End If End Function That's my solution short of having the user manually enter what the format should be. Webbiz
From: Webbiz on 2 Dec 2009 01:03 On Tue, 01 Dec 2009 23:58:48 -0600, Webbiz <nospam(a)noway.com> wrote: >Public Sub SetDataFormat(ByVal Digits As Long) > > sDataFormat = "#" > > If Digits > 0 Then sDataFormat = sDataFormat & "." & >Right("###########", Digits) > >End Sub Made a minor change to this. Right("0000000000", Digits) This should force the ending zeroes to show as well. Webbiz
From: Dee Earley on 2 Dec 2009 04:40 On 01/12/2009 20:07, Webbiz wrote: > On Tue, 01 Dec 2009 08:53:00 +0000, Dee Earley > <dee.earley(a)icode.co.uk> wrote: > >> On 01/12/2009 06:48, Webbiz wrote: >>> On Mon, 30 Nov 2009 21:21:52 -0500, "MikeD"<nobody(a)nowhere.edu> wrote: >>>> "Webbiz"<nospam(a)noway.com> wrote >>>>> Private Function DisplayAvgATR(ByRef Ranges() As Single) As String >>> >>> BTW, while you've no problem 'nitpicking'<g>, got any suggestions on >>> a snazy way to set the format pattern of the "#.####" based on the way >>> the values are formatted in the array to begin with? >> >> The values in the array wont have a format as they are Singles. >> Formatting only applies when you convert to a string for display purposes. >> If you want the caller to be able to control the output format, either >> return a single and let it format it or take the format string as a >> parameter. > > > I'm sorry, I didn't explain myself well. > > The data in the array is of a particular format although Singles. > Stock and Commodity prices come in various formats. Here are some > common ones: > <SNIP> > > Actual prices at the time of writing this. :-) > > So when one of these data files are loaded in, the output displays > would need to display results that are also in the appropriate format. But the input values do not have a format by definition of being a Single value. Yes, they will have some amount of decimal places, but no specific "000.00000" format so in your example, the code will ONLY see: 82.625 Live Cattle 59.2 Lean Hogs 399.75 Corn 1059.5 Soybeans 562.5 Wheat If your calling code wants a specific format, it should format the value it gets back (which will also be unformatted) It is still much safer to pass numbers around as numbers rather than formatted strings, then only convert them for display. Another example: a=1.4444444:?a 1.4444444 b=2.0000000:?b 2 c=2.5555556:?c 2.5555556 d=3.0000000:?d 3 ?(a+b+c+d)/4 2.25 You can possibly determine the number of decimal places a number has by multiplying by 10 in a loop until it approaches 0 (Don't use exactly 0 due to floating point maths) -- Dee Earley (dee.earley(a)icode.co.uk) i-Catcher Development Team iCode Systems
From: Jim Mack on 2 Dec 2009 07:33
Webbiz wrote: > Jim Mack wrote: >> >> You are NOT seeing singles. You're seeing strings, which are the >> result of formatting. Accept the default VB formatting, and you get >> what VB decides. Use Format and get what you want. It's really >> pretty simple. > > So what you're saying is that when I place my mouse over the Single > variable and it displays the result of that variable during debug, > it's been converted to a string without the benefit of actually > being assigned to a string? Yes, except that is has been assigned to a string -- a temp string that you have no control over. Any value you see on the screen is a string. -- Jim Mack Twisted tees at http://www.cafepress.com/2050inc "We sew confusion" |