KX Community

Find answers, ask questions, and connect with our KX Community around the world.

Home Forums kdb+ Function Returns unwanted values when a negative value is passed

  • Function Returns unwanted values when a negative value is passed

    Posted by laura on June 11, 2021 at 12:00 am
    The function should return the same negative value as it is passed to it

    Input passed: FormatNumber[enlist -0.331,15]

    Output Should be as “-0.331”

    but my Code output is “-1.699”

     

    FormatNumber:{[col;DecimalPlaces]
    Rows:count col;
    $[DecimalPlaces<=4;afterDecimal:1_`string[floor “j”$(10 xexp DecimalPlaces)*col-floor col)%(10 xexp DecimalPlaces)];[fraction:col-floor col;
    c:where string[col-floor col] like “*e*”;
    col[IndexWithNonZeroDigitAfterFouthDecimalPlace]:col[IndexWithNonZeroDigitAfterFouthDecimalPlace]-floor col[IndexWithNonZeroDigitAfterFouthDecimalPlace];
    afterDecimal:2_`string[(floor “j”$(10 xexp DecimalPlaces)*fraction)%(10 xexp DecimalPlaces) ];
    afterDecimal[IndexWithNonZeroDigitAfterFouthDecimalPlace]:”0”,/:1_afterDecimal[IndexWithNonZeroDigitAfterFouthDecimalPlace];
    ];
    PrecisionERR:0.000000000001;
    Result:({:reverse “,” sv 3 cut reverse string[x] each floor col+PrecisionERR),`”.”,/: ?[0=count each DecimalPoint:Rows#enlist[DecimalPlaces#”0″]; afterDecimal];
    :?[DecimalPlaces>0;Result;ssr[;”.”;””]each Result]
    };

     

    The Above code doesnot work for negative value.

    laura replied 1 month, 2 weeks ago 2 Members · 1 Reply
  • 1 Reply
  • mklee

    Member
    June 19, 2021 at 12:00 am

    U used , rather than ;

    Have u looked into .Q.fmt

     

     

Log in to reply.