BasicMicro - Forums

www.basicmicro.com
It is currently Mon May 21, 2012 7:54 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: Float and the LCD
PostPosted: Tue May 27, 2003 10:00 am 
RegSel con p8
RdWrPin con p9
CLK con p10
DB4 con p11
DB5 con p12
DB6 con p13
DB7 con p14

aa var long
bb var long
cc var long

aa = float 100.01
bb = float 200.02
cc = aa fadd bb
LCDINIT RegSelCLKDB7DB6DB5DB4, RdWrPin
LCDWRITE RegSelCLKDB7DB6DB5DB4, RdWrPin,[TWOLINE,CLEAR] ' Set for 2 line display and clears the screen
PAUSE 100 ' Give the display time to react
LCDWRITE RegSelCLKDB7DB6DB5DB4, ["Math test"]
LCDWRITE RegSelCLKDB7DB6DB5DB4, [SCRRAM



Top
  
 
 Post subject: Re: Float and the LCD
PostPosted: Tue May 27, 2003 10:00 am 
Ken,

The problem is that 0.01 cannot be represented exactly in binary floating point. Rounded off to 2 places, the result is 300.03 (apparently what the debugger is showing). Truncated to 3 places, the result is 300.029 (what your LCD shows).

Don

[This message has been edited by donblake (edited 05-27-2003).]



Top
  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group

phpBB SEO