Computer Vision as a Tool for Correlation Analysis of Images of the Microstructure of Bi-Te-based Thermoelectric Materials
DOI:
https://doi.org/10.63527/1607-8829-2025-4-64-75Keywords:
thermoelectric materials, semiconductors, Bi-Te, artificial intelligence, machine learning, computer vision, defect segmentation, multimodal learning, deep learning, modeling, kinetic coefficients, atomic force microscopy, optical metallographic studiesAbstract
This paper presents the concept of a multimodal computer vision and machine learning tool for establishing correlations between the surface microstructure of extruded Bi-Te-based thermoelectric materials and their thermoelectric properties. The platform integrates data from atomic force microscopy (AFM), metallographic microscopy, chemical composition, thermoelectric properties, and synthesis process conditions into a single database for further deep learning. The developed tool for creating an annotated database, a neural network architecture for automated defect segmentation, and a multimodal data fusion model for predicting thermoelectric properties are described.
References
1. Snyder G. J., & Toberer E. S. (2008). Complex thermoelectric materials. Nature Materials, 7(2), 105–114. https://doi.org/10.1038/nmat2090
2. Pei Y., Shi X., LaLonde A., Wang H., Chen L., & Snyder G. J. (2011). Convergence of electronic bands for high performance bulk thermoelectrics. Nature, 473(7345), 66–69. https://doi.org/10.1038/nature09996
3. Kim S. I., Lee K. H., Mun H. A., Kim H. S., Hwang S. W., Roh J. W., Yang D. J., Shin W. H., Li X. S., Lee Y. H., Snyder G. J., & Kim S. W. (2015). Dense dislocation arrays embedded in grain boundaries for high-performance bulk thermoelectrics. Science, 348(6230), 109–114. https://doi.org/10.1126/science.aaa4166
4. DeCost B. L., & Holm E. A. (2015). A computer vision approach for automated analysis and classification of microstructural image data. Computational Materials Science, 110, 126–133. https://doi.org/10.1016/j.commatsci.2015.08.011
5. Azimi S. M., Britz D., Engstler M., Fritz M., & Mücklich F. (2018). Advanced steel microstructural classification by deep learning methods. Scientific Reports, 8(1). https://doi.org/10.1038/s41598-018-20037-5
6. Wang Z., Yokoyama Y., Onda T., Adachi Y., & Chen Z. (2019). Improved thermoelectric properties of hot‐extruded Bi–Te–Se bulk materials with Cu doping and property predictions via machine learning. Advanced Electronic Materials, 5(6). https://doi.org/10.1002/aelm.201900079
7. Wang Z., Adachi Y., & Chen Z. (2019). Processing optimization and property predictions of hot‐extruded Bi–Te–Se thermoelectric materials via machine learning. Advanced Theory and Simulations, 3(1). https://doi.org/10.1002/adts.201900197
8. Wang Z.-L., Funada T., Onda T., & Chen Z.-C. (2023). Knowledge extraction and performance improvement of Bi2Te3-based thermoelectric materials by machine learning. Materials Today Physics, 31, 100971. https://doi.org/10.1016/j.mtphys.2023.100971
9. Sheng Y., Deng T., Qiu P., Shi X., Xi J., Han Y., & Yang J. (2021). Accelerating the discovery of Cu–Sn–S thermoelectric compounds via high-throughput synthesis, characterization, and machine learning-assisted image analysis. Chemistry of Materials, 33(17), 6918–6924. https://doi.org/10.1021/acs.chemmater.1c01856
10. Ling J., Hutchinson M., Antono E., DeCost B., Holm E. A., & Meredig B. (2017). Building data-driven models with microstructural images: generalization and interpretability (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1711.00404
11. Pei Z., Rozman K. A., Doğan Ö. N., Wen Y., Gao N., Holm E. A., Hawk J. A., Alman D. E., & Gao M. C. (2021). Machine‐learning microstructure for inverse material design. Advanced Science, 8(23). https://doi.org/10.1002/advs.202101207
12. Pregowska A., Roszkiewicz A., Osial M., & Giersig M. (2024). How scanning probe microscopy can be supported by Artificial Intelligence and quantum computing (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2406.19397
13. Zhang X., et al. (2013). Nano-thermoelectric Seebeck coefficient in-situ quantitative characterization device based on AFM. WO Patent WO2013189111A1.
14. Moon, J., et al. (2021). Compliant three-dimensional thermoelectrics. US Patent US20210175406A1.
15. Russell B. C., Torralba A., Murphy K. P., & Freeman W. T. (2007). LabelMe: A database and Web-based tool for image annotation. International Journal of Computer Vision, 77(1–3), 157–173. https://doi.org/10.1007/s11263-007-0090-8
16. Zhu J.-Y., Park T., Isola P., & Efros A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks (Version 7). arXiv. https://doi.org/10.48550/ARXIV.1703.10593
17. Ronneberger O., Fischer P., & Brox T. (2015). U-Net: convolutional networks for biomedical image segmentation (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1505.04597
18. He K., Zhang, X. Ren S., & Sun J. (2015). Deep residual learning for image recognition (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1512.03385
19. Oktay O., Schlemper J., Folgoc L. L., Lee M., Heinrich M., Misawa K., Mori K., McDonagh S., Hammerla N. Y., Kainz B., Glocker B., & Rueckert D. (2018). Attention U-Net: learning where to look for the pancreas (Version 3). arXiv. https://doi.org/10.48550/ARXIV.1804.03999
20. Lin T.-Y., Dollar P., Girshick R., He K., Hariharan B., & Belongie S. (2017). Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 936–944). 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2017.106
21. Fotos G., Campbell A., Murray P., & Yakushina E. (2023). Deep learning enhanced Watershed for microstructural analysis using a boundary class semantic segmentation. Journal of Materials Science, 58(36), 14390–14410. https://doi.org/10.1007/s10853-023-08901-w
22. Sudre C. H., Li W., Vercauteren T., Ourselin S., & Cardoso M. J. (2017). Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations. arXiv. https://doi.org/10.48550/ARXIV.1707.03237
23. Lin T.-Y., Goyal P., Girshick R., He K., & Dollar P. (2017). Focal loss for dense object detection. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 2999–3007). 2017 IEEE International Conference on Computer Vision (ICCV). IEEE. https://doi.org/10.1109/iccv.2017.324
24. Yaslioglu M. M. (2025). Attention is all you need until you need retention (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2501.09166
25. Chen T., Kornblith S., Norouzi M., & Hinton G. (2020). A simple framework for contrastive learning of visual representations (Version 3). arXiv. https://doi.org/10.48550/ARXIV.2002.05709
26. Selvaraju R. R., Cogswell M., Das A., Vedantam R., Parikh D., & Batra D. (2016). Grad-CAM: Visual explanations from deep networks via gradient-based localization. arXiv. https://doi.org/10.48550/ARXIV.1610.02391
27. Lundberg S., & Lee S.-I. (2017). A unified approach to interpreting model predictions (Version 2). arXiv. https://doi.org/10.48550/ARXIV.1705.07874
