Resilience Patterns for Data-Intensive Scientific Workflows

This review examines resilience patterns for data-intensive scientific workflows. The organizing question is how workflow engines should preserve scientific meaning through retries, partial failure, and infrastructure change. Ten related scholarly sources are synthesized through a decision-centered framework spanning problem definition, mechanism, measurement, evaluation, implementation, and governance. The review does not invent experiments, pooled estimates, or unreported quantitative results. It instead evaluates the strength and transferability of the available evidence, with particular attention to declaring a workflow successful because every process eventually exited. The resulting framework links technical or empirical performance to explicit use conditions and identifies tests that should precede wider adoption in long-running computational experiments on shared infrastructure.

1. Akkaoui, R., Hei, X., & Cheng, W. (2020). EdgeMediChain: A Hybrid Edge Blockchain-Based Framework for Health Data Exchange. IEEE Access, 8, 113467-113486. https://doi.org/10.1109/access.2020.3003575 DOI
2. Dai, F., Hossain, M. A., & Wang, Y. (2025). State of the Art in Parallel and Distributed Systems: Emerging Trends and Challenges. Electronics, 14(4), 677. https://doi.org/10.3390/electronics14040677 DOI
3. Guedes, T., Jesus, L. A., Ocaña, K. A. C. S., Drummond, L. M. A., & Oliveira, D. D. (2019). Provenance-based fault tolerance technique recommendation for cloud-based scientific workflows: a practical approach. Cluster Computing, 23(1), 123-148. https://doi.org/10.1007/s10586-019-02920-6 DOI
4. McKee, D. W., Clement, S. J., Almutairi, J., & Xu, J. (2018). Survey of advances and challenges in intelligent autonomy for distributed cyber‐physical systems. CAAI Transactions on Intelligence Technology, 3(2), 75-82. https://doi.org/10.1049/trit.2018.0010 DOI
5. Pan, B., Stakhanova, N., & Ray, S. (2023). Data Provenance in Security and Privacy. ACM Computing Surveys, 55(14s), 1-35. https://doi.org/10.1145/3593294 DOI
6. Pouya, I., Pronk, S., Lundborg, M., & Lindahl, E. (2016). Copernicus, a hybrid dataflow and peer-to-peer scientific computing platform for efficient large-scale ensemble sampling. Future Generation Computer Systems, 71, 18-31. https://doi.org/10.1016/j.future.2016.11.004 DOI
7. Salloum, S., Dautov, R., Chen, X., Peng, P. X., & Huang, J. Z. (2016). Big data analytics on Apache Spark. International Journal of Data Science and Analytics, 1(3-4), 145-164. https://doi.org/10.1007/s41060-016-0027-9 DOI
8. Senger, H., & Geyer, C. (2016). Parallel and distributed computing for Big Data applications. Concurrency and Computation Practice and Experience, 28(8), 2412-2415. https://doi.org/10.1002/cpe.3813 DOI
9. Versluis, L., & Iosup, A. (2021). A survey of domains in workflow scheduling in computing infrastructures: Community and keyword analysis, emerging trends, and taxonomies. Future Generation Computer Systems, 123, 156-177. https://doi.org/10.1016/j.future.2021.04.009 DOI
10. Zhang, Z., Barbary, K., Nothaft, F. A., Sparks, E. R., Zahn, O., Franklin, M. J., Patterson, D. A., & Perlmutter, S. (2016). Kira: Processing Astronomy Imagery Using Big Data Technology. IEEE Transactions on Big Data, 6(2), 369-381. https://doi.org/10.1109/tbdata.2016.2599926 DOI