BasicMicro - Forums

www.basicmicro.com
It is currently Wed Feb 22, 2012 1:53 pm

Board index » Atom, AtomNano, AtomPro » AtomPro

All times are UTC - 8 hours [ DST ]



Post new topic Reply to topic  Page 1 of 1
  [ 2 posts ] 
  Print view Previous topic | Next topic 
Author Message
KurtEck
 Post subject: SQR function fails on very large numbers...
PostPosted: Thu Jan 26, 2012 4:39 pm 
Offline
Master

Joined: Tue Nov 21, 2006 9:34 am
Posts: 505
I have been playing around with a Lynxmotion ARM and had some wild swings... Traced down to SQR function failing...
If you run this simple program:
Code:
v1   var slong
v2   var slong

v1 = 1063250000
v2 = 1117250000

pause 500
serout s_out, i38400, [ dec SQR(v1), " ", dec SQR(v2), 13]

The output is:
Code:
32607 657


This is on build 2...15 of studio

Kurt
Top
 Profile  
 
Acidtech
 Post subject: Re: SQR function fails on very large numbers...
PostPosted: Sat Jan 28, 2012 7:34 pm 
Offline
Site Admin
User avatar

Joined: Thu Mar 01, 2001 11:00 am
Posts: 801
Location: Temecula, CA
I seem to recall SQR is limited to less than 4^32. But I don't remember off the top of my head what the limit is. Send me an email, Kurt and I'll look at it on Monday. A temporary work around would be to use fsqr for now. It's a lot slower but it should have no limit on the input number.

_________________
Nathan Scherdin
Basic Micro - Robotic Technology Evolved
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 2 posts ] 

Board index » Atom, AtomNano, AtomPro » AtomPro

All times are UTC - 8 hours [ DST ]

Who is online

Users browsing this forum: No registered users and 1 guest

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group

phpBB SEO