add low book confidence detector

[mshannon-sil]

This adds a function to detect low book confidence. It will be used in SILNLP to flag books with low confidence, and it will be ported to Machine so Serval can use it. It's quite straightforward, so let me know if there's any additional logic we should be adding to machine.py.

---

This change is 

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.02%. Comparing base (2d9f122) to head (098f83b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #350   +/-   ##
=======================================
  Coverage   92.01%   92.02%           
=======================================
  Files         387      389    +2     
  Lines       24458    24485   +27     
=======================================
+ Hits        22505    22532   +27     
  Misses       1953     1953           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pmachapman]

Just one question!

@pmachapman reviewed 3 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on Enkidu93 and mshannon-sil).

---

tests/quality_estimation/test_book_confidence.py line 30 at r1 (raw file):

def test_is_book_confidence_unusually_low_nan() -> None:
    with raises(ValueError):
        is_book_confidence_unusually_low(float("nan"))

Right now NmtEngineBuildJob._train_model() returns a confidence value of float("nan"). Do we want to handle this as a special not defined case (and so ignore), or do you think it should be an error case as per your implementation?

Code quote:

def test_is_book_confidence_unusually_low_nan() -> None:
    with raises(ValueError):
        is_book_confidence_unusually_low(float("nan"))

[mshannon-sil]

tests/quality_estimation/test_book_confidence.py line 30 at r1 (raw file):

Previously, pmachapman (Peter Chapman) wrote…

Right now NmtEngineBuildJob._train_model() returns a confidence value of float("nan"). Do we want to handle this as a special not defined case (and so ignore), or do you think it should be an error case as per your implementation?

Would this function ever see the output of NmtEngineBuildJob._train_model(), seeing as how this function is only meant to run on book confidence scores?

I'm good with doing whatever is most consistent with how the machine/machine.py repo has been handling nan. @Enkidu93 any thoughts?

[mshannon-sil]

I added one more change so that the method matches the current function stub's signature in Serval.

[Enkidu93]

@Enkidu93 partially reviewed 3 files and made 2 comments.
Reviewable status: 1 of 3 files reviewed, 1 unresolved discussion (waiting on mshannon-sil and pmachapman).

---

tests/quality_estimation/test_book_confidence.py line 30 at r1 (raw file):

Previously, mshannon-sil wrote…

Would this function ever see the output of NmtEngineBuildJob._train_model(), seeing as how this function is only meant to run on book confidence scores?

I'm good with doing whatever is most consistent with how the machine/machine.py repo has been handling nan. @Enkidu93 any thoughts?

Yeah, I don't think we need to worry about this since the confidence coming from _train_model is a different sort of confidence value relevant for SMT.