Improving Model Performance

Newton is a foundation model capable of addressing a wide variety of tasks in the physical space, offering strong general-purpose performance by default. However, you may be able to achieve better results for your use case by customizing the model. Customizing Newton involves systematically improving your results through adaptation, evaluation, and optimization. This guide provides recommendations to help get you started with this process.

​

Prepare the Data

Before you begin, you need to prepare the data for inference. The steps involved in data preparation vary depending on the type of data being analyzed and the method being used to analyze it. There are several ways in which data may need to be prepared prior to inference:

• Convert data: Data needs to be converted into the required input format.
  • Machine State Classification tasks require CSV files.
  • Activity Detection tasks require JSONL files.
• Label data: Make sure each column of CSV files is labeled. Ensure JSONL files have useful field names.
• Generate N-shot files (for Machine State Classification tasks): Create N-shots that demonstrate the states you wish to detect.
• Impute missing data: Sequences of missing values (NaNs) shorter than the window length should be imputed (for example, estimated using linear interpolation). Missing value sequences longer than the window length will return NaN outputs from the model.
• Normalize values: The best normalization strategy is dependent on the data and should be chosen as part of your optimization policy. Possible strategies include per-window normalization, per-session (file) normalization, and global normalization, each per variate. Per-window normalization can be implemented using the platform configuration parameter normalize_input = True.
  If normalizing globally, the normalization coefficients should be determined using the N-shot/validation source data and then applied to the test data.
• Remove unneeded and unused variates: Remove any variates whose values do not change across the dataset. Use only variates that are present in all files.

​

Assess Newton

The next step is to assess Newton’s out-of-the-box performance for your target task. Start by testing the model on a representative sample of your real-world data. Experiment with different prompting styles, response formats, and instruction phrasing to observe how the model behaves under different conditions. Specific things to consider in this stage:

• Instruction clarify and specificity
• Output formatting requirements. For example, do you prompt for structured or conversational outputs?
• Input length and context complexity
• Edge cases and ambiguous examples

The goal during this stage is to build an understanding of the model’s baseline capabilities when analyzing your data, and what limitations you encounter.

• Where does the model already perform well without customization?
• What types of input produce inconsistent results?
• Are failures caused by reasoning errors, unexpected formatting, factual errors, or missing context?
• Does the model reliably follow instructions?

​

Determine Success Criteria

Whether the initial results are strong or otherwise, assembling a high-quality evaluation dataset is one of the most important investments you can make during the optimization process. Optimizing the model with data that doesn’t adequately represent the kind of input it will see in real-world use doesn’t result in a model tuned to work well for your data. A well-designed evaluation set provides a stable reference point for measuring improvements, comparing prompt strategies, and tracking regressions. Without a good evaluation set, any optimizations you make become subjective and difficult to validate. Before you start to optimize the model, it’s critical to clearly define what “good performance” means for your use task or use case. Your success criteria should be quantitative whenever possible to ensure consistent and objective evaluation. Key metrics to consider include:

• Accuracy: How often the model’s predictions match the correct expected results (i.e., ground truth).
• Precision: The quality and accuracy of the model’s positive predictions.
• Recall: The completeness of the model and its ability to find all actual positive instances within the data.
• False positive rate: The percentage of time in which the model incorrectly claims negative instances in the data are positive.
• Exact match rate: The percentage of time in which the model produces results that exactly match ground truth.

Your choices for these metrics should be made based on the real-world impact of errors and should be aligned with your business and/or operational requirements.

​

Develop Quality Evaluation Examples

Once you’ve determined your success criteria, construct a set of examples that serve as a ground truth benchmark for model performance. These so-called “golden examples” should include representative inputs that reflect standard operational scenarios, as well as variations in conditions, edge cases, adversarial inputs, and noisy or incomplete data. Each example must be paired with high-quality expected outputs or ground truth labels that clearly define the correct model behavior for that case. The goal is often quality over quantity; a small but carefully-curated dataset is often more valuable than a large but low-quality dataset. High-quality evaluation sets prioritize coverage, correctness, and realism over volume, ensuring that even a limited number of examples can reliably expose model weaknesses and guide meaningful improvements.

​

Prompt Tuning

Many tasks can be significantly improved through better prompting alone. Prompt tuning should be approached systematically, leveraging the evaluation set of examples to measure the impact of different prompt versions. Things to consider when optimizing prompts:

• Provide sufficient context: The model performs best when it has enough contextual information to properly interpret the input data and operational setting. Useful context may include detailed descriptions of sensors of instrumentation, operating conditions, data collection methodology, units and expected signal behavior, known system constraints, and relevant domain terminology. Providing rich contextual grounding often improves both reasoning quality and output consistency, especially for specialized industrial or scientific tasks.
• Clear task definition: Prompts should explicitly describe the objective of the task, the expected type of response, the decision criteria, and any output constraints or assumptions the model should follow. Clearly defining what constitutes a successful answer helps to reduce ambiguity and improves consistency across different inputs and operating conditions.
• Include tips and guidance: For complex inputs, it’s often beneficial to guide the model toward the most relevant signals or patterns in the data. This may include highlighting important sensor correlations, expected temporal behaviors, anomaly signatures, operational thresholds, or known indicators of failure modes. Directing the model’s attention toward meaningful aspects of the input can significantly improve reasoning accuracy and reduce irrelevant or inconsistent outputs.
• Optimize response format: Structured outputs simplify downstream integration, validation, and automation by enforcing consistent response formats such as JSON schemas, tables, templates, or classification labels. However, overly restrictive output constraints can sometimes reduce the model’s flexibility and reasoning capabilities. In some cases, letting the model provide intermediate reasoning or explanations before generating the final structured response can improve both accuracy and interpretability.

​

Fine Tuning

While prompt engineering can solve many use cases effectively, some applications require a deeper level of adaptation. Fine-tuning lets Newton internalize task-specific behaviors, domain knowledge patterns, and output conventions directly into the model weights. Fine-tuning is the process of continuing training on a pre-trained foundation model using a curated dataset tailored to a specific task or domain. Instead of teaching the model from scratch, fine-tuning adapts the existing capabilities of Newton toward a narrower operational objective. The objective isn’t to replace the foundation model’s general intelligence, but to specialize and stabilize it for a particular operational environment. The challenge of fine-tuning comes from building a high-quality fine-tuning dataset. The quality of the dataset is usually the single largest factor determining fine-tuning success. A smaller, cleaner dataset often outperforms a much larger but noisy dataset. Key principles when designing your fine-tuning dataset:

• Variety over quantity: Choosing a smaller dataset with diverse and representative examples is usually more valuable than a very large dataset with repetitive patterns. Diversity helps the model generalize better to unseen inputs.
• Coverage over all cases: Training data should include the full range of expected scenarios, including edge cases, ambiguous inputs, failures, and uncommon situations. Be sure not to include only the most frequent examples.
• Balanced distribution: Avoid over-representation of certain classes, formats, or workflows. Providing a balanced dataset prevents the model from becoming biased toward dominant patterns, and improves overall reliability.
• Build iteratively: Fine-tuning datasets should evolve over time. As new failure nodes and production issues are discovered, additional examples should be added to the fine-tuning dataset to continuously improve model performance.