• kdb+

  Function Returns unwanted values when a negative value is passed

  Posted by Laura on June 11, 2021 at 12:00 am

  The function should return the same negative value as it is passed to it

  Input passed: FormatNumber[enlist -0.331,15]

  Output Should be as “-0.331”

  but my Code output is “-1.699”

   

  FormatNumber:{[col;DecimalPlaces]
  Rows:count col;
  $[DecimalPlaces<=4;afterDecimal:1_`string[floor “j”$(10 xexp DecimalPlaces)*col-floor col)%(10 xexp DecimalPlaces)];[fraction:col-floor col;
  c:where string[col-floor col] like “*e*”;
  col[IndexWithNonZeroDigitAfterFouthDecimalPlace]:col[IndexWithNonZeroDigitAfterFouthDecimalPlace]-floor col[IndexWithNonZeroDigitAfterFouthDecimalPlace];
  afterDecimal:2_`string[(floor “j”$(10 xexp DecimalPlaces)*fraction)%(10 xexp DecimalPlaces) ];
  afterDecimal[IndexWithNonZeroDigitAfterFouthDecimalPlace]:”0”,/:1_afterDecimal[IndexWithNonZeroDigitAfterFouthDecimalPlace];
  ];
  PrecisionERR:0.000000000001;
  Result:({:reverse “,” sv 3 cut reverse string[x] each floor col+PrecisionERR),`”.”,/: ?[0=count each DecimalPoint:Rows#enlist[DecimalPlaces#”0″]; afterDecimal];
  :?[DecimalPlaces>0;Result;ssr[;”.”;””]each Result]
  };

   

  The Above code doesnot work for negative value.

  Laura replied 8 months ago 2 Members · 1 Reply
• 1 Reply

• 

  mklee

  Member

  June 19, 2021 at 12:00 am

  U used , rather than ;

  Have u looked into .Q.fmt

   

   

Log In to Reply