讨论 POC 可行性验证Discussing a Proof of Concept

Sam, I have your draft POC plan. Before I approve sample collection, I want to make sure the test will be fair and repeatable.

That is essential. The draft has two targets: identifying the selected component for each product variant, and confirming the sequence between installation and the tool event.

The line has more operations. Why are they not included?

They are important, but including them now would introduce different camera views and process rules. We have listed them as future scope so the first result remains interpretable.

How will you divide the material?

We will use one labelled group for development and keep a separate group unavailable to model tuning. The held-back group will be used only for the agreed test.

We should agree on a random or stratified sampling rule before anyone sees the model results. Our process owner can confirm the labels, but should not hand-pick cases because of an expected outcome.

Agreed. We can stratify by the approved product variants and observation conditions, then sample randomly within each group. Your process owner confirms the business label, and the held-back set stays unavailable to model tuning.

Some exception cases are rare. Do we have to create them on the line?

Only if the recreation is safe, approved and does not risk a real product. Otherwise, we can use a controlled offline setup or document that the case remains outside the current evidence.

The plan mentions a baseline review. What decision follows that review?

Both teams review whether the target is observable and whether the result categories are unambiguous. Then we freeze the test definition, including the sample register and how each outcome is counted.

Who runs the final test?

Our engineer can run the configured test, while representatives from both teams observe. The result log and reviewed evidence will be shared, and any disputed case will be set aside for joint review rather than silently reclassified.

Will the test send any command to the line controller?

No. This POC is observation-only. It may read an approved product identifier, but it will not write a control command or block production.

What happens if the lighting changes during the test?

We record the condition and compare it with the agreed test range. If it falls outside that range, we do not hide the result; we report it separately as evidence of a limitation or a need for further testing.

Then the final report should include the scope, sample register, configuration, results, disputed cases and known limitations.

Exactly. It will also state what the POC supports as a next decision and what still requires production engineering. That keeps the conclusion useful without overstating it.

I want the sampling rule to be clear in the plan. How will we decide the strata without creating groups that favour an easy result?

We should derive the strata from known process factors agreed before model review, such as approved product variant and observation condition. We should not create a new group simply because its model result looks different.

Who performs the random selection within each stratum?

The method can be run from a jointly reviewed sample register with a recorded seed or equivalent auditable procedure. Your process owner then confirms the labels without seeing or changing the model outcome.

What if a selected frame is unusable because the target is completely outside the approved view?

The exclusion rule must be written before selection and applied consistently. We record the excluded item and reason, then use the same predefined replacement method. We do not quietly replace a difficult but valid case.

How will you handle a case where our label reviewers disagree?

The case enters a documented adjudication process led by the authorised process owner. Until the business label is resolved, it stays outside the scored result and appears in the report as an unresolved-label case.

Can the model or threshold be adjusted after we see the held-back results?

Not within the same scored run. Any change creates a new version and requires a new, independent test set. Otherwise, the held-back material effectively becomes development material and no longer provides an unbiased check.

The POC also includes workflow sequence. How do we test timing without depending on a live controller?

We can use a reviewed event simulator and recorded test sequences. The simulator must have documented timestamps and expected outcomes. It supports logic validation, but the report will state that real-controller behaviour remains unverified.

What if the first result is below the agreed rule?

We do not relabel or remove valid cases. We analyse the failure modes, document whether they arise from observability, data coverage or the method, and decide whether another development cycle is justified.

Then the decision is not simply pass or fail; it may be proceed, redesign the observation setup or stop that target.

Correct. A disciplined POC turns uncertainty into evidence for a decision. Its value comes from transparent scope, controlled sampling and honest limits, not from forcing every target to appear successful.

I will update the approval note to name the sampling rule, label owner, exclusion rule and adjudication path. That should make the final review easier to audit.

Thank you. I will align the test script and report template with those controls, so the method described before testing matches the evidence presented afterwards.