Adding token IDs and increasing precision in `--debug` mode

[alex-ie]

Describe the Issue
In --debug mode lines with generated tokens are printed. However, only selected token have ID printed, the rest - only text value - consider adding IDs for others too. Also, for several tokens one of which is close to 100% (which is often) the output contains 100%, 0%, 0% - that looks incorrect - when I changed precision to 4 digits those started to have meaningful %s.

The change in code is not large * if it is to become default. If you believe current more compact output is preferable as default (e.g. to maintain backward compatibility), please consider adding new positional argument value for --debugmode [DEBUGMODE] to activate extended output. Additionally I propose to increase hard limit of tokens from 4 to 5 (or if this will be non-default mode - bypass hard limit, but about that I am not sure - how many tokens could there be in logprobs_max?).

If you agree that adding new value of positional argument will be beneficial, but have no time for that now, I am ready to try to make changes myself and do pull request.

TIA

Additional Information:
v1.112

*
Three lines in:
gpttype_adapter.cpp:

printf("(%s <%d> %.2f%%)", RemoveBell(topstr).c_str(), toppick.selected_tokenid, toppick.selected_probability*100);
with
printf("(%s <%d> %.4f%%)", RemoveBell(topstr).c_str(), toppick.selected_tokenid, toppick.selected_probability*100);

printf("(%s %.2f%%)", RemoveBell(tokenizedstr).c_str(), toppick.p[i]*100);
with
printf("(%s <%d> %.4f%%)", RemoveBell(tokenizedstr).c_str(), toppick.tokenid[i], toppick.p[i]*100);

int maxtoshow = (toppick.tokenid.size()>4?4:toppick.tokenid.size()); //hardcode limit even if we have more logprobs_max
with
int maxtoshow = (toppick.tokenid.size()>5?5:toppick.tokenid.size()); //hardcode limit even if we have more logprobs_max

Resulting in:

                if(debugmode==1 && !is_quiet && top_picks_history.size()>0)
                {
                    printf(" [");
                    bool firstloop = true;
                    TopPicksData toppick = top_picks_history[top_picks_history.size()-1];
                    std::string topstr = toppick.selected_token;
                    ::utreplace(topstr, "\n", "\\n");
                    printf("(%s <%d> %.4f%%)", RemoveBell(topstr).c_str(), toppick.selected_tokenid, toppick.selected_probability*100);
                    int maxtoshow = (toppick.tokenid.size()>5?5:toppick.tokenid.size()); //hardcode limit even if we have more logprobs_max
                    for (int i=0;i<maxtoshow;++i)
                    {
                        if(toppick.tokenid[i]==toppick.selected_tokenid)
                        {
                            continue;
                        }
                        printf(" ");
                        std::string tokenizedstr = toppick.tokens[i];
                        ::utreplace(tokenizedstr, "\n", "\\n");
                        printf("(%s <%d> %.4f%%)", RemoveBell(tokenizedstr).c_str(), toppick.tokenid[i], toppick.p[i]*100);
                    }
                    printf("]\n");
                }

[LostRuins]

Hmm... the problem is the terminal will quickly become cluttered by all the additional IDs.

If the logprobs are important, why not retrieve them from the API? That provides far more detailed information, including the byte codes and ID of all top candidates

check out the last_logprobs API.

[alex-ie]

Ok, I will check API, thanks!