Patterns (Cell Press) 2022 | 食品知识图谱在食品科学和工业中的应用

导语

近日，我们食品计算小组关于食品知识图谱的综述工作“Applications of knowledge graphs for the food science and industry” (Weiqing Min, Chunlin Liu, Leyi Xu, Shuqiang Jiang*)发表在Patterns (Cell Press) 上。Patterns是Cell Press旗下的学术期刊，旨在发表数据科学领域突破性的原创研究，分享计算机科学、生命科学和社会科学等跨学科的数据科学解决方案，促进跨学科研究者之间的交流合作。

各种网络(如物联网和移动网络)、数据库(如营养表和食品成分数据库)和社交媒体(如Instagram和Twitter)的快速发展产生了海量食品数据，为使用数据驱动的计算方法研究食品科学和工业中的问题提供了前所未有的机遇。然而，这些多源异构的食品数据往往呈现信息孤岛特点，导致其难以被充分利用。知识图谱以结构化的形式提供了统一和标准化的概念体系，可以有效组织这些食品数据以便进行广泛应用。本文首先介绍了包括食品本体和食品知识图谱在内的食品知识组织方法。然后总结了食品知识图谱的代表性应用，如新配方开发、饮食疾病相关性研究和个性化饮食推荐等。最后，本文讨论了该领域的未来研究方向，如构建多模态食品知识图谱和人类健康食品知识图谱等。由于其在表示和推理方面的能力，食品知识图谱将推动食品互联网(Internet of Food)和食品智能的发展，所蕴含的巨大潜力将吸引越来越多的学者将知识图谱应用于食品科学和工业领域。

论文链接：

https://www.sciencedirect.com/science/article/pii/S2666389922000691

课题组主页：

http://123.57.42.89/FoodComputing__Home.html

引言

食品在人类生活中起到了至关重要的作用。食品从农场经过种植、收获、包装、加工、转化、生产、运输、分配到消费，形成了食品系统[1]，这一过程产生了大量的多学科和多源的异构食品数据（如营养成分表、健康数据库、食品图像和食谱）。这些数据使得计算机技术和数字技术成为了食品科学和工业中不可或缺的一部分，也促进了食品科学和工业的发展。目前，人工智能技术已经应用在食品过程建模[2,3]，食品质量评估[4]，食品对象识别与分析[5-7]，食品认证与溯源[8]和膳食评估[9,10]等各种与食品相关的应用中。然而，这些食品数据仍然难以满足许多任务对数据有效共享、组织和可追溯的需求。例如，在食品供应链系统中，来自不同利益相关者的数据可能遵循不同的规范，这限制了供应链中食品术语的对齐和对不同数据源信息的融合，使得食品供应系统管理和优化的难度大大增加[11]。此外，食品污染可追溯性和暴露评估等相对复杂的问题涉及到多个领域的数据，这对食品系统融合和组织食品数据及从多源食品数据中提取食品知识的能力提出了要求[12]。因此，有必要使用一种统一术语组织食品数据，以提取食品相关信息和融合食品知识，使研究人员、食品制造商、食品分销商、零售商和消费者等不同利益相关者均能从中受益。食品本体可以用于统一食品术语，基于食品本体的食品知识图谱可以进一步建模现实世界的食品实例，并支持相关任务。这使得本体和知识图谱能够有效组织食品科学和工业中的食品数据[13,14]。

食品本体

不同的食品本体重点关注食品的不同方面。具体可以分为以下四种不同类型：(1)烹饪和食谱本体，(2)营养和健康本体，(3)食品子领域本体，以及(4)全面的食品本体。 (1)烹饪和食谱本体(如Taaable[17]，Cooking Ontology[18]，Edamam Food Ontology和 BBC Food Ontology)主要面向烹饪行为、食物、食谱和厨具等数据，用于融合烹饪和食谱信息。 (2) 营养和健康本体关注健康和营养概念。该类本体能够起到提供饮食与营养摄入的监测和建议(如PIPS Food Ontology[19], FOODS Ontology[20]和HeLiS[21])、协助饮食推荐(如Unified Traveler and Nutrition Ontology[22])和支持营养科学研究(如ONS[23]和OFFF[24])的作用。 (3)食品子领域本体侧重于刻画具体的食品类别(如为海鲜加工设计的Seafood Ontology[25]和适配肉类供应链领域的MESCO[26])或具体的食品研究领域(如关注食品公共安全问题的FSO[27])。 (4)全面的食品本体(如FoodOn[28]和 RICHIFIELDS Ontology[29])包含全面的食品概念，这使得它们能够服务于更广泛的应用，如构建全面的食品知识图谱或融合不同来源的食品数据。

食品知识图谱

类似的，我们将食品知识图谱分为四种不同的类型，包括 (1)食谱知识图谱，(2)营养和健康知识图谱，(3)食品安全知识图谱，以及 (4)全面的食品知识图谱。 (1) 食谱知识图谱主要服务于与食谱相关的应用。比如，Foodbar Knowledge Graph[30]被用于推荐食物及提供食物相关的描述性分析服务；RcpKG[31]则能够根据个人喜好和社会关系给出合理的食谱推荐。 (2) 营养和健康知识图谱可应用于健康相关的饮食应用，如CFKG[32]被用在食物的语义搜索中，FoodKG[33]和HDKG[34]则被用于实现健康食品的推荐以及食物成分替换。此外，也有一些包含了心血管疾病知识的食品知识图谱被用于支持疾病相关的研究[35]。 (3) 食品安全知识图谱有助于构建食品安全相关的查询和智能问答系统，能够提取食品抽查信息，帮助人们获取不合格食品信息[36,37]。 (4) 全面的知识图谱(如面向食物、能源和水的FEW，侧重于农业和饮食的AgriKG [38]和研究文化的World Food Atlas[39]) 旨在收录更广泛的食品概念，协助开发各式各样的食品应用和涉及食品知识的其他领域应用。这些知识图谱被用于探索食物、农业、健康和文化之间的关系，推动跨领域研究的进一步开展。

应用

1 新食品研发

利用食品知识图谱的知识组织能力和推理能力，可以开发基于现有产品的新式食品。Shirai等人提出了一种启发式方法，能够基于食品知识图谱FoodKG中食品成分属性和食品成分名称潜在语义的相似性，对替代成分进行排序[40,41]，并选择最佳替代成分。在食品行业，这种方法可以用于寻找现有产品的成分替代品，降低食品生产成本，或是开发新的食物口味，以增加产品多样性。对于消费者而言，这种基于食品知识图的成分替代方法可以用于满足其个性化的定制需求。此外，还可以借助知识图谱开发包含成分和烹饪步骤在内的新食谱[42]。

2 食品问答系统

知识图谱支持使用机器语言的信息查询，因此只要构建将自然语言问题转换为机器查询语句的程序，就可以开发有关食谱和营养问题的问答系统，实现营养信息查询、营养成分比较和符合特定标准的食谱搜索[43]。此类系统还可以用于回答食品有关领域(如营养与疾病)的问题，例如“怎样增加蛋糕的纤维含量？”或“可以用什么代替巧克力蛋糕中的牛奶？”。

3 饮食-疾病关联分析

饮食与疾病的相关性研究一直是食品科学和营养学的重要问题[44-46]。通过构建食品知识图谱，我们可以建立疾病、饮食、食物、原料和化学成分之间的联系，然后对其相关性进行深入分析。已经有NutriChem，DietRx等工作从大量文献中收集了有关饮食、营养和疾病的知识，并将它们融合在统一的本体下，这使得我们可以很容易地了解某些类型的食物对人体的作用，进而可以推断出哪些类型的食物是有害的或有益的，为理解饮食对健康的影响提供知识基础[35]。

4 视觉食品分析

无损、快速、可靠地分析食品质量对于食品工业具有重要意义。视觉食品分析是无损分析方法中最具前景的方法之一，已被运用在部分食品应用中。来自食品知识图谱的丰富知识(例如成分及其关系)可以作为先验知识被引入，并通过机器学习算法进行矢量化表示。知识图谱中有关食材共存和食物-食材关系的知识可以被转换为共生矩阵，进而辅助预测菜肴的类别，提高视觉食物识别性能[47]。虽然到目前为止，尚未有工作使用食品知识图谱来处理更复杂的食物视觉分析任务（如食物对象检测和分割），但类似的思想已在许多场景中得到运用。

5 个性化食品推荐

为实现个性化食品和营养推荐，需要使用全面的个人信息生成饮食建议或推荐，这个过程中会涉及到大量营养和饮食数据的融合。食品知识图谱提供了食品相关术语和知识的标准和统一表示，可以提高食物推荐系统的精度，增加推荐系统的多样性并为其带来更好的可解释性[48]。当与个人知识图谱结合（如PHKG）时，食品知识图谱还可以利用其中有关个人健康的信息和个人饮食的行为记录，为用户量身定制更加个性化的饮食推荐[49-51]。

6 食品供应链管理

食品知识图谱可以在食品供应链管理中对食品数据进行建模和融合。利用其统一表示的特点，食品知识图谱可以确定一个术语在特定条件下的确切含义，并将食品加工中的操作属性对齐，统一食品供应链中涉及的每个部分，进而支持更可靠的回溯和相关查询。例如，FTTO为不同的食品设计了不同的标准化属性，因此不同的食品状态可以通过加工流程联系起来，并在其生命周期中保持一致性。这样，食品供应链的工作人员可以在加工流程中进行智能查询，并追溯过程中所有的安全风险。MESCO[26]沿用了FTTO中的溯源方式和食品相关概念，并收录了更多肉类供应链中的特有概念。

7 食品装备智能制造

食品知识图谱还可以用于构建智能厨房或智能工业设备，对物联网不同传感器收集的数据进行融合和挖掘并用于进一步的智能决策，构建自动化食品工业生产线和面向消费者的智能终端设备[52]。在工业上，工业机器人还能够从食品知识图谱中做出更智能的决策，它们可以通过传感器访问加工状态，获取食品材料的物理特性，并根据食品知识图谱中的知识进行智能加工控制。

未来方向

1 多模态食品知识图谱

构建多模态食品知识图谱可将面向食物的多模态学习技术应用到更多任务中，例如跨模态食谱-食物图像检索和生成[53-55]，基于图像自动生成食谱及支持面向食品的多模态对话系统。

2 基于食品知识图谱的推理

使用食品知识图谱的第一步是对食品知识进行表示并进行复杂的推理，但在降低推理复杂性上目前尚无较好方法。改进复杂推理有助于进一步改善知识图谱所支持的复合食物关系预测[56]和食物推荐[57]等任务的性能。

3 食品大数据组织与挖掘

近年来，在全球范围内产生了海量的食品大数据[58]。使用食品知识图谱将能够更好地组织和利用这些数据，进而推动食品、营养、健康和法规的相关研究[59]，帮助分析以往的食品安全事故[60,61]以及实现基于大数据的精准营养推荐[62]等。

4 食联网(IoF)

来自不同来源和硬件标准的数据往往缺少集成手段，这会影响IoF的构建。食品知识图谱可以提供关于食品信息的标准，融合不同来源的大规模食品数据，让机器、消费者和公司能够轻松访问食品相关信息，从而推动更多应用的开发[63,64]。

5 人类健康

食品知识图谱为利用大规模、结构化的食品知识建立健康监测平台提供了技术方法。使用食品知识图谱作为该平台的核心模块，可以支持饮食行为的跟踪和监测，健康相关的搜索和推荐及食品相关的营养、饮食和疾病研究。

6 食品智能

食品计算[65]已经将食品与健康、文化和医学等方面建立了联系，有着广泛应用。利用食品知识图谱可以提高已有食品计算方法的性能，如图像识别[66,67]、目标检测[68]，问答系统[69]及决策和推理[70]，这将大大促进食品科学和工业中各种研究和应用的开展。

参考文献

[1] Global Panel on Agriculture and Food Systems for Nutrition Future food systems: for people, our planet, and prosperity. Technical report Glob. Panel Agric. Food Syst. Nutr. (2020) [2] G. V. S. Bhagya Raj and Kshirod K. Dash. Comprehensive study on applications of artificial neural network in food process modeling. Critical Reviews in Food Science and Nutrition, pages 1–28, 2020. [3] Qing Sun, Min Zhang, and Arun S. Mujumdar. Recent developments of artificial intelligence in drying of fresh food: A review. Critical Reviews in Food Science and Nutrition, 59(14):2258–2275, 2019. [4] Hanieh Amani, Katalin Kerti Badakné, and Amin Mousavi Khaneghah. Current progress in the utilization of smartphone-based imaging for quality assessment of food products: A review. Critical Reviews in Food Science and Nutrition, pages 1–13, 12 2020. [5] Simon Knez and Luka Šajn. Food object recognition using a mobile device: Evaluation of currently implemented systems. Trends in Food Science & Technology, 99:460–471, 2020. [6] Marcus Klasson, Cheng Zhang, and Hedvig Kjellström. Using variational multi-view learning for classification of grocery items. Patterns, 1(8):100143, 2020. [7] Yao Liu, Hongbin Pu, and Da-Wen Sun. Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices. Trends in Food Science & Technology, 113:193–204, 2021. [8] Ning Liang, Sashuang Sun, Chu Zhang, Yong He, and Zhengjun Qiu. Advances in infrared spectroscopy combined with artificial neural network for the authentication and traceability of food. Critical Reviews in Food Science and Nutrition, pages 1–22, 2020. [9] Quin Thames, Arjun Karpur, Wade Norris, Fangting Xia, Liviu Panait, Tobias Weyand, and Jack Sim. Nutrition5k: Towards automatic nutritional understanding of generic food. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8903–8911, 2021. [10] Larissa Oliveira Chaves, Ana Luiza Gomes Domingos, Daniel Louzada Fernandes, Fabio Ribeiro Cerqueira, Rodrigo Siqueira-Batista, and Josefina Bressan. Applicability of machine learning techniques in food intake assessment: A systematic review. Critical Reviews in Food Science and Nutrition, pages 1–18, 2021. [11] N. N. Misra, Yash Dixit, Ahmad Al-Mallahi, Manreet Singh Bhullar, Rohit Upadhyay, and Alex Martynenko. IoT, big data and artificial intelligence in agriculture and food industry. IEEE Internet of Things Journal, pages 1–1, 2020. [12] Jianping Qian, Bingye Dai, Baogang Wang, Yan Zha, and Qian Song. Traceability in food processing: Problems, methods, and performance evaluations—a review. Critical Reviews in Food Science and Nutrition, pages 1–14, 2020 [13] Heiko Paulheim. Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic Web, 8(3):489–508, 2017. [14] Haussmann Steven, Chen Yu, Seneviratne Oshani, Rastogi Nidhi, Codella James, Chen Ching-Hua, McGuinness Deborah, and J. Zaki Mohammed. FoodKG enabled Q&A application. In Proceedings of International Semantic Web Conference, 2019. [15] Allan M Collins and M Ross Quillian. Retrieval time from semantic memory. Journal of verbal learning and verbal behavior, 8(2):240–247, 1969. [16] Amit Singhal. Introducing the knowledge graph: Things, not strings, 2012. [17] Amélie Cordier, Valmi Dufour-Lussier, Jean Lieber, Emmanuel Nauer, Fadi Badra, Julien Cojan, Emmanuelle Gaillard, Laura Infante-Blanco, Pascal Molli, Amedeo Napoli, and Hala Skaf-Molli. Taaable: A case-based system for personalized cooking. Studies in Computational Intelligence, 494:121–162, 01 2014. [18] Fernando Batista, J. Paulo, Nuno Mamede, H. Vaz, and Ricardo Ribeiro. Ontology construction: Cooking domain. Artificial Intelligence: Methodology, Systems, and Applications, 4183:213–221, 01 2006. [19] Jaime Cantais, David Dominguez, Valeria Gigante, Loredana Laera, and Valentina Tamma. An example of food ontology for diabetes control. In Proceedings of the International Semantic Web Conference 2005 workshop on Ontology Patterns for the Semantic Web, 2005. [20] C. Snae and M. Bruckner. FOODS: A food-oriented ontology-driven system. In IEEE International Conference on Digital Ecosystems and Technologies, pages 168–176, 2008 [21] Mauro Dragoni, Tania Bailoni, Rosa Maimone, and Claudio Eccher. HeLiS: An ontology for supporting healthy lifestyles. In The Semantic Web – International Semantic Web Conference 2018, pages 53–69, 2018. [22] Shakir Karim, Umair Shaikh, Quratulain Rajput, and Zaheeruddin Asif. Ontology-based personalized dietary recommendation for travelers. In Southern association for information systems conference, 2015. [23] Francesco Vitali, Rosario Lombardo, Damariz Rivero, Fulvio Mattivi, Pietro Franceschi, Alessandra Bordoni, Alessia Trimigno, Francesco Capozzi, Giovanni Felici, Francesco Taglino, Franco Miglietta, Nathalie De Cock, Carl Lachat, Bernard De Baets, Guy De Tré, Mariona Pinart, Katharina Nimptsch, Tobias Pischon, Jildau Bouwman, Duccio Cavalieri, and the ENPADASI consortium. ONS: An ontology for a standardized description of interventions and observational studies in nutrition. Genes & Nutrition, 13(1):12, 2018. [24] Muhammad Amith, Chidinma Onye, Tracey Ledoux, Grace Xiong, and Cui Tao. The ontology of fast food facts: conceptualization of nutritional fast food data for consumers and semantic web applications. BMC medical informatics and decision making, 21(7):1–16, 2021. [25] Teresa Pizzuti, Giovanni Mirabelli, Giovanni Grasso, and Giulia Paldino. MESCO (meat supply chain ontology): An ontology for supporting traceability in the meat supply chain. Food Control, 72:123–133, 2017. [26] Li Qin, Zhigang Hao, and Liang Zhao. Food safety knowledge graph and question answering system. In Proceedings of International Conference on Information Technology: IoT and Smart City, pages 559–564, 2019. [27] Vinu Sherimon, PC Sherimon, Alaa Ismaeel, Winny Varkey, and B Naveen. Modeling of seafood domain using ontology. International Journal of Open Information Technologies, 9(2):65–69, 2021. [28] Damion M. Dooley, Emma J. Griffiths, Gurinder S. Gosal, Pier L. Buttigieg, Robert Hoehndorf, Matthew C. Lange, Lynn M. Schriml, Fiona S. L. Brinkman, and William W. L. Hsiao. FoodOn: A harmonized food ontology to increase global food traceability, quality control and data integration. npj Science of Food, 2, 2018. [29] Tome Eftimov, Gordana Ispirova, Peter Korošec, and Barbara Seljak. The RICHFIELDS framework for semantic interoperability of food information across heterogenous information systems. In Proceedings of International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, volume 1, pages 315–322, 09 2018. [30] Unai Zulaika, Asier Gutiérrez, and Diego López-de-Ipiña. Enhancing profile and context aware relevant food search through knowledge graphs. In International Conference on Ubiquitous Computing and Ambient Intelligence, volume 2, page 1228, 2018. [31] Zhenfeng Lei, Anwar Ul Haq, Adnan Zeb, Md Suzauddola, and Defu Zhang. Is the suggested food your desired?: Multi-modal recipe recommendation with demand-based knowledge graph. Expert Systems with Applications, 186:115708, 2021. [32] Yang Chi, Congcong Yu, Xiaohui Qi, and Hao Xu. Knowledge management in healthcare sustainability: A smart healthy diet assistant in traditional Chinese medicine culture. Sustainability, 10(11), 2018. [33] Haussmann Steven, Chen Yu, Seneviratne Oshani, Rastogi Nidhi, Codella James, Chen Ching-Hua, McGuinness Deborah, and J. Zaki Mohammed. FoodKG enabled Q&A application. In Proceedings of International Semantic Web Conference, 2019. [34] Lan Huang, Congcong Yu, Yang Chi, Xiaohui Qi, and Hao Xu. Towards smart healthcare management based on knowledge graph technology. In Proceedings of the International Conference on Software and Computer Applications, page 330–337, 2019. [35] Soodabeh Milanlouei, Giulia Menichetti, Yanping Li, Joseph Loscalzo, Walter C Willett, and Albert-László Barabási. A systematic comprehensive longitudinal evaluation of dietary factors associated with acute myocardial infarction and fatal coronary heart disease. Nature communications, 11(1):1–14, 2020. [36] Li Qin, Zhigang Hao, and Liang Zhao. Food safety knowledge graph and question answering system. In Proceedings of International Conference on Information Technology: IoT and Smart City, pages 559–564, 2019 [37] Li Qin, Zhigang Hao, and LiPing Yang. Question answering system based on food spot-check knowledge graph. In Proceedings of International Conference on Computing and Data Engineering, ICCDE 2020, page 168–172, 2020 [38] Yuanzhe Chen, Jun Kuang, Dawei Cheng, Jianbin Zheng, Ming Gao, and Aoying Zhou. AgriKG: An agricultural knowledge graph and its applications. In Database Systems for Advanced Applications, pages 533–537, 2019. [39] Ali Rostami, Zhouhang Xie, Akihisa Ishino, Yoko Yamakata, Kiyoharu Aizawa, and Ramesh Jain. World Food Atlas Project. In Proceedings of International Workshop on Multimedia for Cooking and Eating Activities, pages 33–36, 2021. [40] Sola Shirai, Oshani Seneviratne, Minor Gordon, Ching-Hua Chen, Deborah L McGuinness, Nidhi Rastogi, et al. Semantics-driven ingredient substitution in the FoodKG. In The Semantic Web – International Semantic Web Conference, volume 2721, pages 242–247, 2020. [41] Sola S. Shirai, Oshani Seneviratne, Minor E. Gordon, Ching-Hua Chen, and Deborah L. McGuinness. Identifying ingredient substitutions using a knowledge graph of food. Frontiers in Artificial Intelligence, 3, 2021. [42] Florian Pinel, Lav R. Varshney, and Debarun Bhattacharjya. A Culinary Computational Creativity System, pages 327–346. Atlantis Press, Paris, 2015. [43] Jayne V Woodside, Ian S Young, and Michelle C McKinley. Fruit and vegetable intake and risk of cardiovascular disease. Proceedings of the Nutrition Society, 72(4):399–406, 2013. [44] Ashkan Afshin, Patrick John Sur, Kairsten A. Fay, Leslie Cornaby, Giannina Ferrara, et al. Health effects of dietary risks in 195 countries, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. The Lancet, 393(10184):1958–1972, 2019. [45] Zhiyun Zhao, Mian Li, Chao Li, Tiange Wang, Yu Xu, Zhizheng Zhan, Weishan Dong, Zhiyong Shen, Min Xu, Jieli Lu, Yuhong Chen, Shenghan Lai, Wei Fan, Yufang Bi, Weiqing Wang, and Guang Ning. Dietary preferences and diabetic risk in China: A large-scale nationwide Internet data-based study. Journal of Diabetes, 12(4):270–278, 2020. [46] James Joseph, Greg Cole, Elizabeth Head, and Donald Ingram. Nutrition, brain aging, and neurodegeneration. The Journal of neuroscience : the official journal of the Society for Neuroscience, 29(41):12795–12801, 2009. 35 Food Knowledge Graph [47] Jingjing Chen, Liangming Pan, Zhipeng Wei, Xiang Wang, ChongWah Ngo, and Tatseng Chua. Zero-shot ingredient recognition by multi-relational graph convolutional network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 10542–10550, 2020. [48] Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. RippleNet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of ACM International Conference on Information and Knowledge Management, pages 417–426, 2018. [49] Chang-Shing Lee, Mei-Hui Wang, Huan-Chung Li, and Wen-Hui Chen. Intelligent ontological agent for diabetic food recommendation. In IEEE International Conference on Fuzzy Systems, pages 1803–1810, June 2008. [50] Piyaporn Tumnark, Leandro Oliveira, and Nonchai Santibutr. Ontology-based personalized dietary recommendation for weightlifting. In Proceedings of International Workshop on Computer Science in Sports, 2013. [51] Vanesa Espín, María V. Hurtado, Manuel Noguera, and Kawtar Benghazi. Semantic-based recommendation of nutrition diets for the elderly from agroalimentary thesauri. In Flexible Query Answering Systems, pages 471–482, 2013. [52] Jamshed Iqbal, Zeashan Hameed Khan, and Azfar Khalid. Prospects of robotics in food industry. Food Science and Technology, 37:159–165, 2017 [53] Javier Marín, Aritro Biswas, Ferda Ofli, Nicholas Hynes, Amaia Salvador, Yusuf Aytar, Ingmar Weber, and Antonio Torralba. Recipe1M+: A dataset for learning cross-modal embeddings for cooking recipes and food images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1):187–203, 2021. [54] Dim P. Papadopoulos, Youssef Tamaazousti, Ferda Ofli, Ingmar Weber, and Antonio Torralba. How to make a pizza: Learning a compositional layer-based GAN model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7994–8003, 2019. [55] Hao Wang, Guosheng Lin, Steven C. H. Hoi, and Chunyan Miao. Structure-aware Generation Network for recipe generation from images. In European Conference on Computer Vision, volume 12372, pages 359–374, 2020. [56] Donghyeon Park, Keonwoo Kim, Seoyoon Kim, Michael Spranger, and Jaewoo Kang. FlavorGraph: A largescale food-chemical graph for generating food representations and recommending food pairings. Scientific Reports, 11(1), 2021. [57] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4–24, 2021. [58] Dandan Tao, Pengkun Yang, and Hao Feng. Utilization of text mining as a big data analysis tool for food science and nutrition. Comprehensive Reviews in Food Science and Food Safety, 19(2):875–894, 2020. [59] Heather Greenfield and David AT Southgate. Food Composition Data: Production, Management, and Use. Food & Agriculture Org., 2003. [60] Shanquan Chen, Dandan Huang, Wenyan Nong, and Hoi Shan Kwan. Development of a food safety information database for Greater China. Food Control, 65:54–62, 2016. [61] Hans J. P. Marvin, Esmée M. Janssen, Yamine Bouzembrak, Peter J. M. Hendriksen, and Martijn Staats. Big data in food safety: An overview. Critical Reviews in Food Science and Nutrition, 57(11):2286–2295, 2017. [62] Stavros Bashiardes, Anastasia Godneva, Eran Elinav, and Eran Segal. Towards utilization of the human genome and microbiome for personalized nutrition. Current Opinion in Biotechnology, 51:57–63, 2018. [63] Hoang Long Nguyen, Dang Thinh Vu, and Jason J. Jung. Knowledge graph fusion for smart systems: A survey. Information Fusion, 61:56–70, 2020. [64] Nicholas M. Holden, Eoin P. White, Matthew. C. Lange, and Thomas L. Oldfield. Review of the sustainability of food systems and transition using the Internet of Food. npj Science of Food, 2(1):18, 2018. [65] Weiqing Min, Shuqiang Jiang, Linhu Liu, Yong Rui, and Ramesh Jain. A survey on food computing. ACM Computing Surveys, 52(5):92:1–92:36, 2019. [66] K. Marino, R. Salakhutdinov, and A. Gupta. The more you know: Using knowledge graphs for image classification. In IEEE Conference on Computer Vision and Pattern Recognition, pages 20–28, 2017. [67] Simon Knez and Luka Šajn. Food object recognition using a mobile device: Evaluation of currently implemented systems. Trends in Food Science & Technology, 99:460–471, 2020. [68] Yuan Fang, Kingsley Kuan, Jie Lin, Cheston Tan, and Vijay Chandrasekhar. Object detection meets knowledge graphs. In Proceedings of International Joint Conference on Artificial Intelligence, pages 1661–1667, 2017. [69] Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander Smola, and Le Song. Variational reasoning for question answering with knowledge graph. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, pages 6069–6076, 2018. [70] Xiaojun Chen, Shengbin Jia, and Yang Xiang. A review: Knowledge reasoning over knowledge graph. Expert Systems with Applications, 141:112948, 2020.