Traceability back to subset-026

[morido]

Consider lines 39ff in ToStaffResponsible_Conditions.c. Apparently, these conditions are based on 4.6.2[2].[t]2.[r][7] and 4.6.3[2].[t]*. However, I cannot see any hint for those origins in the source. Is there a chance to include the respective traceability data as comments in the generated files?

Apart from this, I would expect the actual conditions, as they are mentioned in the spec (i.e. the stuff below 4.6.3[2].[t]*), to be modelled in separate functions since the are meant to be reused for other mode transitions as well. However, in lines 39ff they are all inlined. Was this a deliberate design choice or does the KCG perform such kinds of "optimizations"?

[JakobGartner]

As a general remark, we have (nobody really has) not used the commenting features of SCADE fully that also allow to propagate comments to code.
If we had for example used the RM gateway and linked everything with the SRS, we could also propagate traceability info as comments to the C- code

This remark is independently of the question how much sense this would make in a model- based V&V and certification approach which would require integration of SCADE based development and V&V process into a to-be-defined overall process.

(Cert Kit needed)

On 09 Aug 2015, at 02:47, Moritz Dorka [email protected] wrote:

Consider lines 39ff in ToStaffResponsible_Conditions.c

srcAndBinary/Green openETCS Non-Vital Demonstrator/Source Code/Source Code ETCS Onboard Unit System/Generated_Code/ToStaffResponsible_Conditions.c

Line 39 in 434737f

outC->Condition44 = (Current_Level == M_LEVEL_Level_1) & Level_prev_L0_LNTC &

. Apparently, these conditions are based on 4.6.2[2].[t]2.[r][7] and 4.6.3[2].[t]*. However, I cannot see any hint for those origins in the source. Is there a chance to include the respective traceability data as comments in the generated files?

The source is the model. The C- code is only an intermediate step. If you need traceability to the code, it will be primarily with with the model

Apart from this, I would expect the actual conditions, as they are mentioned in the spec (i.e. the stuff below 4.6.3[2].[t]*), to be modelled in separate functions since the are meant to be reused for other mode transitions as well. However, in lines 39ff they are all inlined. Was this a deliberate design choice or does the KCG perform such kinds of "optimizations"?

The level of direct traceability between model and code depends on the KCG options.
It is up to the user to select the appropriate options.
KCG does no optimisations by itself.
However, all levels of optimisations are covered by KCG certification, meaning that it is trusted to produce identical behaviour of the generated code vs. the model.

I did not check, but probably line 39ff are inlined beaches the use chose “expansion” as a KCG option.
If this is the case, this will however be commented.

However I do not agree with some of the statements. We are not “thinking code” but “thinking model”. I do not see a need for 1:1 mapping between SRS (subsub) paragraphs and model level functions.
There is also no hard link between functional breakdown on model and code levels as KCG can be instructed to either generate everything in one function, many functions, one file, many files.

This would have been topic of a process which has not been defined upfront.

—
Reply to this email directly or view it on GitHub #3.

[morido]

If we had for example used the RM gateway and linked everything with the SRS, we could also propagate traceability info as comments to the C- code

Oh. So you do not do that...?

We are not “thinking code” but “thinking model”.

While I generally do understand this attitude, I believe for someone without access to SCADE (me, for instance) checking out the generated code is about the only viable way to get an idea of what you are currently doing.
Hence, from an "open proofs" point of view I would like to see the C-code contain as much useful info as possible.

The source is the model. The C- code is only an intermediate step. If you need traceability to the code, it will be primarily with with the model

So you are essentially saying: The KCG is conformant with [insert list of safe software standards] in their highest levels. Therefore, its output always behaves in the exact same way as its input and there is no need for traceability.

Why does the KCG produce code in the first place, then? An executable binary would be just fine.
And secondly: If ever there is an accident whose cause can be attributed to a certain line of code (confer with Ariane 5), how would you trace that back into the model?

I do not see a need for 1:1 mapping between SRS (subsub) paragraphs and model level functions.

Well, but how do you know if some required functionality is already in the model (perhaps redundantly) or still missing? I.e. how do you track your spec coverage? And how do you think verification will work without this?

[BaseliyosJacob]

Dear Moritz,

the traceability is a very important item in terms of SIL 4 certification. An i fully agree that following the "open proof" concept it should be respected as well the full transparency.

In the current project status we are aware that this part is missing due to the lack of ressources. Since we shift all priorities to completentess of the ETCS System Model Utrecht - Amsterdam proof of concept.

This was a project consoritum decision.

That means that we are aware of open gaps for the full traceability. Therefore we will be prepared to close n this gaps in a follow up project (Shift2Rail) since we are allocated all our remaining ressources in the openETCS project to completete functionality until end of this year.

Therefore i consider your input as a very valuable and should be documentated as open issues and work for the follow up project. Since the follow up project will definitevely focus on a SIL 4 certifiable Corridor A prototype. We really appreciate your support according to our current preparation of contenct in the follow up project.

Thank you very much

[JakobGartner]

On 09 Aug 2015, at 12:47, Moritz Dorka [email protected] wrote:

If we had for example used the RM gateway and linked everything with the SRS, we could also propagate traceability info as comments to the C- code

Oh. So you do not do that...?

Not everybody does it, not everybody has the license.

Of course it is also possible to annotate the model directly with remarks and comments that get generated as comments into the C- code, but for lack of resources we don’t do it systematically even if it is a requirement expressed from the project leader.

We are not “thinking code” but “thinking model”.

While I generally do understand this attitude, I believe for someone without access to SCADE (me, for instance) checking out the generated code is about the only viable way to get an idea of what you are currently doing.
Hence, from an "open proofs" point of view I would like to see the C-code contain as much useful info as possible.

I understand that. It is however not obvious as a task.
However, this is a very old discussion which exists as long as SCADE.
I agree that comments and traceability (which then are propagated to documentation, as well as to code, are important, and I hope there will be time to enrich the models that way.

By the way, you can always use SCADE without a license as long as you just review the models.

The source is the model. The C- code is only an intermediate step. If you need traceability to the code, it will be primarily with with the model

So you are essentially saying: The KCG is conformant with [insert list of safe software standards] in their highest levels. Therefore, its output always behaves in the exact same way as its input and there is no need for traceability.

Well, the need for traceability may come from requirements derived from the standards. That is why SCADE comes with everything to provide that, and with a very sophisticated approach on verification and verification of verification. It is just that in openETCS, we hardly manage to get an executable spec (which is the SCAE model). We are far from running through a complete process.
Why does the KCG produce code in the first place, then? An executable binary would be just fine.

SCADE generates target agnostic C (or Ada or SPARK) code because it needs to be portable and target- independent.

Actually, the SCADE simulator generates a binary…. (SCADE simulation is a software- in- the- loop simulator)

And secondly: If ever there is an accident whose cause can be attributed to a certain line of code (confer with Ariane 5), how would you trace that back into the model?

I do not see a need for 1:1 mapping between SRS (subsub) paragraphs and model level functions.

Well, but how do you know if some required functionality is already in the model (perhaps redundantly) or still missing? I.e. how do you track your spec coverage? And how do you think verification will work without this?

What I am trying to say is that this kind of coverage is done at model level.
Code generation is then yet another step.

Granularity of model is not necessarily the same as SRS granularity. There may be N:N relationships, not just 1:1.
Granularity of code is not necessarily the same as model granularity. We have, for example, now have introduced a higher level of optimisation in order to successfully scale simulation (which does not only include the EVC but also track, DMI and environment)

If we would go to a vital system, the setting would have to be selected according to the requirements imposed by the applicable norm and process.
I agree with you that in such a case, traceability (to code) may be of higher importance.

SCADE itself does not impose a specific process. It is the other way round.

Maybe you know the attached document, I was a co-author of the first versions. It is of course not a SW development plan in the sense of CENELEC, but gives a high-level introduction to using SCADE in a EN50128 process.

Jakob

—
Reply to this email directly or view it on GitHub #3 (comment).

[JakobGartner]

Moritz,

to invest your time in a fruitful way (not trying to find your way through code that has indeed, as Baseliyos said, not been generated from a model that is well- linked to the requirements due to project priorities), can I propose to take a selected component and run the exercise of a review (including traceability)?

More concretely: We select a module, and work constructively until you are satisfied with all your questions, as far as possible in the time/ resource contrasts we have ?

If we select the module “small enough”, but representative it could be a good input to the follow- on project.

On 09 Aug 2015, at 12:47, Moritz Dorka [email protected] wrote:

If we had for example used the RM gateway and linked everything with the SRS, we could also propagate traceability info as comments to the C- code

Oh. So you do not do that...?

We are not “thinking code” but “thinking model”.

While I generally do understand this attitude, I believe for someone without access to SCADE (me, for instance) checking out the generated code is about the only viable way to get an idea of what you are currently doing.
Hence, from an "open proofs" point of view I would like to see the C-code contain as much useful info as possible.

The source is the model. The C- code is only an intermediate step. If you need traceability to the code, it will be primarily with with the model

So you are essentially saying: The KCG is conformant with [insert list of safe software standards] in their highest levels. Therefore, its output always behaves in the exact same way as its input and there is no need for traceability.

Why does the KCG produce code in the first place, then? An executable binary would be just fine.
And secondly: If ever there is an accident whose cause can be attributed to a certain line of code (confer with Ariane 5), how would you trace that back into the model?

I do not see a need for 1:1 mapping between SRS (subsub) paragraphs and model level functions.

Well, but how do you know if some required functionality is already in the model (perhaps redundantly) or still missing? I.e. how do you track your spec coverage? And how do you think verification will work without this?

—
Reply to this email directly or view it on GitHub #3 (comment).

[morido]

The motivation behind my initial question was certainly not to blame you for all sorts of weaknesses in your model / code. Rather, I am still trying to compile suitable project artifacts for a scientific article on our toolchain.
To find and comprehensibly link such artifacts (i.e. to turn that article into a compelling story) I am dependant on a working traceability infrastructure. So it is, indeed, a little disappointing to learn this does not yet exist.

More concretely: We select a module, and work constructively until you are satisfied with all your questions, as far as possible in the time/ resource contrasts we have ?

Well, if you have the resources to do this, it would be more than welcome. Judging from @BaseliyosJacob's recent statements, I was of the opinion that WP3 is working at its limits and there is absolutely no time to assist anyone concerned with less important tasks.

[JakobGartner]

Yes we don’t feel blamed, I just felt that trying to understand code generated from SCADE requires to start from the model.
I have seen so many people trying to reverse- understand SCADE and its really not obvious.

I can ask @Mairamou to assist you, she is not funded by the project and could spare some part of her time on helping you. As we want to invest anyway in setting up a SIL4 process (as part of our exploitation efforts) it is a sensible investment.

She will however need some guidance as her background on certification processes is zero.
She has however proven to be a very quick learner.

There are also some parts of the model that are more mature (and better annotated) than others. You could consider starting there.
If not, I would propose a component from TrackAtlas (I have to see which one is a good fit).

I would also have wished to work more on higher- level and process questions in order to get much closer to SIL4 objectives, but we are all stuck in trying to compensate for the resources that have already been used in earlier phases/ or that just broke away.

On 09 Aug 2015, at 15:48, Moritz Dorka [email protected] wrote:

The motivation behind my initial question was certainly not to blame you for all sorts of weaknesses in your model / code. Rather, I am still trying to compile suitable project artifacts for a scientific article on our toolchain.
To find and comprehensibly link such artifacts (i.e. to turn that article into a compelling story) I am dependant on a working traceability infrastructure. So it is, indeed, a little disappointing to learn this does not yet exist.

More concretely: We select a module, and work constructively until you are satisfied with all your questions, as far as possible in the time/ resource contrasts we have ?

Well, if you have the resources to do this, it would be more than welcome. Judging from @BaseliyosJacob https://github.com/BaseliyosJacob's recent statements, I was of the opinion that WP3 is working at its limits and there is absolutely no time to assist anyone concerned with less important tasks.

—
Reply to this email directly or view it on GitHub #3 (comment).

[UweSteinkeFromSiemens]

Please note, that calculateTrainPosition already has been linked with the SRS via the Scade Requirements Gateway / Reqtify by using the requirements generated by @morido tools. So, there is at least this example.

[JakobGartner]

Hello Uwe,

thanks!
When I wrote

"There are also some parts of the model that are more mature (and better annotated) than others. You could consider starting there. “, I meant calculateTrainPosition.
I only did not know whether you would have time/resource to help Moritz, so I left the question open.

On 10 Aug 2015, at 10:16, Uwe Steinke [email protected] wrote:

Please note, that calculateTrainPosition already has been linked with the SRS via the Scade Requirements Gateway / Reqtify by using the requirements generated by @morido https://github.com/morido tools. So, there is at least this example.

—
Reply to this email directly or view it on GitHub #3 (comment).

[T12z]

for my 50ct on this,
agreeing on the above,
and there are also packages such as SpeedSupervision that do contain quite some traceability / reqtify information, but that are still anything but trivial to dive into 😇

I also found it is hard to add traceability when a single line of specification maps into a multitude of operators and on the other hand whether top/wrapping operators should receive traces for the sake of understanding. But I also guess this will be one of the outcomes of this project to see how stric you have to formalize a process and coding once contributors are from a multitude environments and hierachies.

[morido]

@UweSteinkeFromSiemens, @T12z Thanks for your hints on packages which contain links to their respective requirements. However, I believe this information does not survive KCG's transformation into C. At least none of the files that I checked contained anything useful. Specifically I looked into:

• calculateTrainpositionAttributes_CalculateTrainPosition_Pkg.c
• calculateTrainPositionInfo_CalculateTrainPosition_Pkg.c
• calculateTrainPosition_CalculateTrainPosition_Pkg.c
• SpeedSupervision_Integration_SpeedSupervision_Integration_Pkg.c

Do not hate me for still "thinking code", instead of "thinking model" here. But my intention is to trace a line of C code back to a requirement - which is why this issue is in the [srcAndBinary] repository rather than in [modelling].
Any chance the KCG can be reconfigured to dump some more meaningful data into the comments here?

---

@T12z

I also found it is hard to add traceability when a single line of specification maps into a multitude of operators and on the other hand whether top/wrapping operators should receive traces for the sake of understanding.

Well, I guess you would need a properly defined process which addresses such issues. The status quo of not adding traceability information at all because of this is certainly the worst of all possible solutions. Ultimately this may severely limit the reusability of your model for any follow-up projects / industrial applications.

---

Side note 1:
I assume the filename pattern for the KCG generated files goes somewhat like this:
SUBUNIT_UNIT_PACKAGE.[ch]. However, searching in an alphabetically sorted directory listing would become a lot easier if that pattern was PACKAGE_UNIT_SUBUNIT.[ch].

Side note 2:
@JakobGartner Github does not seem to support attaching files to issues via mail. So the attachment you mentioned in the end of your comment here got lost. Perhaps, you can make this available in a different way?

[JakobGartner]

the trick is to tick the box “progagate to c” at the Notes/ Comments properties settings

On 10 Aug 2015, at 13:31, Moritz Dorka [email protected] wrote:

@UweSteinkeFromSiemens https://github.com/UweSteinkeFromSiemens, @T12z https://github.com/T12z Thanks for your hints on packages which contain links to their respective requirements. However, I believe this information does not survive KCG's transformation into C. At least none of the files that I checked contained anything useful. Specifically I looked into:

calculateTrainpositionAttributes_CalculateTrainPosition_Pkg.c https://github.com/openETCS/srcAndBinary/blob/master/Green%20openETCS%20Non-Vital%20Demonstrator/Source%20Code/Source%20Code%20ETCS%20Onboard%20Unit%20System/Generated_Code/calculateTrainpositionAttributes_CalculateTrainPosition_Pkg.c
calculateTrainPositionInfo_CalculateTrainPosition_Pkg.c https://github.com/openETCS/srcAndBinary/blob/master/Green%20openETCS%20Non-Vital%20Demonstrator/Source%20Code/Source%20Code%20ETCS%20Onboard%20Unit%20System/Generated_Code/calculateTrainPositionInfo_CalculateTrainPosition_Pkg.c
calculateTrainPosition_CalculateTrainPosition_Pkg.c https://github.com/openETCS/srcAndBinary/blob/master/Green%20openETCS%20Non-Vital%20Demonstrator/Source%20Code/Source%20Code%20ETCS%20Onboard%20Unit%20System/Generated_Code/calculateTrainPosition_CalculateTrainPosition_Pkg.c
SpeedSupervision_Integration_SpeedSupervision_Integration_Pkg.c https://github.com/openETCS/srcAndBinary/blob/master/Green%20openETCS%20Non-Vital%20Demonstrator/Source%20Code/Source%20Code%20ETCS%20Onboard%20Unit%20System/Generated_Code/SpeedSupervision_Integration_SpeedSupervision_Integration_Pkg.c
Do not hate me for still "thinking code", instead of "thinking model" here. But my intention is to trace a line of C code back to a requirement - which is why this issue is in the [srcAndBinary] repository rather than in [modelling].
Any chance the KCG can be reconfigured to dump some more meaningful data into the comments here?

@T12z https://github.com/T12z
I also found it is hard to add traceability when a single line of specification maps into a multitude of operators and on the other hand whether top/wrapping operators should receive traces for the sake of understanding.

Well, I guess you would need a properly defined process which addresses such issues. The status quo of not adding traceability information at all because of this is certainly the worst of all possible solutions. Ultimately this may severely limit the reusability of your model for any follow-up projects / industrial applications.

Side note 1:
I assume the filename pattern for the KCG generated files goes somewhat like this:
SUBUNIT_UNIT_PACKAGE.[ch]. However, searching in an alphabetically sorted directory listing would become a lot easier if that pattern was PACKAGE_UNIT_SUBUNIT.[ch].

Side note 2:
@JakobGartner https://github.com/JakobGartner Github does not seem to support attaching files to issues via mail. So the attachment you mentioned in the end of your comment here #3 (comment) got lost. Perhaps, you can make this available in a different way?

—
Reply to this email directly or view it on GitHub #3 (comment).

[UweSteinkeFromSiemens]

For calculateTrainPosition, the option “progagate to c” is set for some components. I'm not sure, if this serves to propagate notes and traceability information to C code too.