Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Asynchronous publishing in nextflow #719

Open
2 tasks done
DriesSchaumont opened this issue Jun 11, 2024 · 0 comments
Open
2 tasks done

[FEATURE] Asynchronous publishing in nextflow #719

DriesSchaumont opened this issue Jun 11, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@DriesSchaumont
Copy link
Contributor

Feature summary

Currently, the output of a pipeline is only published after the last component of the nextflow workflow has finished.
However, asynchronous publishing is something that could be very beneficial (both from the user perspective, for development and for performance)

Feature description

Allow publishing midway to the pipeline

Why is this feature beneficial?

See above

Alternatives considered

None

Possible solution

Allowing using multiple output channels in emit:

workflow test{
   take:
      foo
   main:
      qc_channel = ...
      final_output  = qc_channel
          | ...
   emit: 
       qc_channel
       final_output
}

Viash would then be able to concat qc_channel and final_output?
concat(test.out)

Confirmation

  • I have searched the existing issues to make sure this is not a duplicate request.
  • I have provided clear and concise information about the feature.
@DriesSchaumont DriesSchaumont added the enhancement New feature or request label Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant