[BUG] Potentially malicious code on commandline detection with PSC v4.3.0

[AndreiBanaru]

Describe the bug

Up until today, we were running Enterprise Security 8.3.0 in Splunk Cloud, with Splunk_ML_Toolkit 5.7.0 and Splunk_SA_Scientific_Python_linux_x86_64 from 4.3.0.

We recently noticed failed jobs for the Potentially malicious code on commandline detection.

The apply query fails with this error:

We immediatelly downgraded to Splunk_ML_Toolkit 5.6.4 and Splunk_SA_Scientific_Python_linux_x86_64 from 4.2.3, and this solved the issue.

As you may already know, starting with ES 8.2.3+, this comes comes with Splunk_ML_Toolkit version 5.6.0 preinstalled.
I believe your suggestion is to use [version 5.5.0],(https://github.com/splunk/security_content/blob/v5.22.0/contentctl.yml#L176) and version 4.2.2, respectively.
But this conflicts with ES 8.2.3+ requirements.

search.log:

03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: Traceback (most recent call last):
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/processors/ApplyProcessor.py", line 169, in apply
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     prediction_df = algo.apply(df, process_options)
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/base.py", line 331, in apply
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     y_hat = self.estimator.predict(X.values)
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/linear_model/_base.py", line 297, in predict
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     return self._decision_function(X)
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:            ~~~~~~~~~~~~~~~~~~~~~~~^^^
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/linear_model/_base.py", line 274, in _decision_function
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     check_is_fitted(self)
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     ~~~~~~~~~~~~~~~^^^^^^
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/utils/validation.py", line 1751, in check_is_fitted
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     tags = get_tags(estimator)
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/utils/_tags.py", line 398, in get_tags
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     tags = estimator.__sklearn_tags__()
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/linear_model/_base.py", line 692, in __sklearn_tags__
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     tags.input_tags.sparse = not self.positive
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:                                  ^^^^^^^^^^^^^
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: AttributeError: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: WARNING Error while applying model "unusual_commandline_detection": 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.218 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: Traceback (most recent call last):
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/processors/ApplyProcessor.py", line 169, in apply
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     prediction_df = algo.apply(df, process_options)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/base.py", line 331, in apply
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     y_hat = self.estimator.predict(X.values)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/linear_model/_base.py", line 297, in predict
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     return self._decision_function(X)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:            ~~~~~~~~~~~~~~~~~~~~~~~^^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/linear_model/_base.py", line 274, in _decision_function
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     check_is_fitted(self)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     ~~~~~~~~~~~~~~~^^^^^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/utils/validation.py", line 1751, in check_is_fitted
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     tags = get_tags(estimator)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/utils/_tags.py", line 398, in get_tags
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     tags = estimator.__sklearn_tags__()
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/4_3_0/lib/python3.13/site-packages/sklearn/linear_model/_base.py", line 692, in __sklearn_tags__
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     tags.input_tags.sparse = not self.positive
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:                                  ^^^^^^^^^^^^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: AttributeError: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: During handling of the above exception, another exception occurred:
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: Traceback (most recent call last):
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/cexc/__init__.py", line 174, in run
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     while self._handle_chunk():
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:           ~~~~~~~~~~~~~~~~~~^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/cexc/__init__.py", line 236, in _handle_chunk
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     ret = self.handler(metadata, body)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/apply.py", line 519, in handler
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     self.controller.execute()
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     ~~~~~~~~~~~~~~~~~~~~~~~^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/chunked_controller.py", line 292, in execute
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     self.processor.process()
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     ~~~~~~~~~~~~~~~~~~~~~~^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/processors/ApplyProcessor.py", line 192, in process
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     self.df = self.apply(self.df, self.algo, self.process_options)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:               ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/Splunk_ML_Toolkit/bin/processors/ApplyProcessor.py", line 177, in apply
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr:     raise RuntimeError(e)
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962199 ChunkedExternProcessorStderrLogger] - stderr: RuntimeError: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.220 ERROR ChunkedExternProcessor [962236 phase_2] - Error in 'apply' command: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.241 INFO  ReducePhaseExecutor [962236 phase_2] - Not downloading remote search.log files.  Reason: Remote log download is currently disabled.
03-02-2026 10:55:46.241 INFO  ReducePhaseExecutor [962236 phase_2] - Not downloading remote search_telemetry.json files. Reason: fetch_remote_search_telemetry=false.
03-02-2026 10:55:46.241 INFO  ReducePhaseExecutor [962236 phase_2] - Ending phase_2
03-02-2026 10:55:46.241 INFO  ReducePhaseExecutor [962222 StatusEnforcerThread] - ReducePhaseExecutor=1 action=QUIT
03-02-2026 10:55:46.241 INFO  DispatchExecutor [962222 StatusEnforcerThread] - Search applied action=QUIT while status=GROUND
03-02-2026 10:55:46.241 INFO  SearchStatusEnforcer [962222 StatusEnforcerThread] - sid=1772448913.3973_1FD5CB9C-2F59-41C7-8872-AA45FE2439F0, newState=FAILED, message=Error in 'apply' command: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.241 ERROR SearchStatusEnforcer [962222 StatusEnforcerThread] - SearchMessage orig_component=SearchStatusEnforcer sid=1772448913.3973_1FD5CB9C-2F59-41C7-8872-AA45FE2439F0 message_key= message=Error in 'apply' command: 'LinearRegression' object has no attribute 'positive'
03-02-2026 10:55:46.241 INFO  SearchStatusEnforcer [962222 StatusEnforcerThread] - State changed to FAILED: Error in 'apply' command: 'LinearRegression' object has no attribute 'positive'

Expected behavior

A clear and concise description of what you expected to happen.

App Version:

• ESCU: v5.22.0
• Splunk Security Essentials: v3.8.3

Additional context

Add any other context about the problem here.

[patel-bhavin]

Hello @AndreiBanaru - We’ve encountered this issue as well, and we’re actively discussing it internally as part of ongoing detection maintenance. At the moment we don’t have a confirmed near-term fix—this appears to be tied to library/runtime compatibility. We’re also evaluating whether these ML-based detections are better handled in Splunk UEBA (given stronger ML capabilities and lifecycle support) and potentially deprecating them in ESCU due to limited maintenance and testing coverage.

Also curious: how extensively are you leveraging the ESCU ML searches in your environment?

[AndreiBanaru]

Hey @patel-bhavin , thanks for acknowledging this.

From the ESCU app, we only have this one active- Potentially malicious code on commandline

[nasbench]

MLTK detections will be deprecated in v5.24.0 - See #3922

Reasoning is - Detection is deprecated as these do not work with the latest Splunk AI Toolkit(5.7.0) and Python for Scientific Computing for Linux 64-bit(4.3.0).

Hence, I will be closing this issue as complete.