Knowlec GraphEmbeddingsand
Federico BIANCHI° Gaetano ROSSIELLO Luca COSTABELLO °Matteo PALMONARI d Pasquale MINERVINIe Bocconi Universiry JBM Research AIdUniversity of Milan-Bicocca Accenture LabsUniversity College London
Abstract Knowledge graph embeddings are now a widely adopted approach toknowledge representatin in which entities and relationships are embeded in vec- tor spaces. In this chapte we introduce the reader to the concept of knowledgegraph embedings by explaining hat hey arc bow they an be geeraed and how they can be evaluated. We summarize the state-of-the-at in this feld by de-scribing the approaches that have been introduced to represent knowledge in the vector space. In relatin to knowledge representation we cosider the problem ofexplainability and discuss models and methods forexlaining predictions obtained via knowledge graph embedings.
Keywords. Knowtledge Graphs. Knowledgc Graph Embeddings Knoledge Repeesentation eXplainable AI
1. Introduetion
A knowledge graph [39] (KG) is an abstraction used in knowledge representation to en-code knowledge in one or more domains by representing entities like Nev York Cityand United States (ie. nodes) and binary relationships that connect these entities; for example Mev York City and United States are connected by the relationshipcontains relationships that connect entities with literals i.e. values from known datastructures such as strings numbers dates and so on; for example a relationship set1edthat connects New York City and the integer 1624 describe a property of the entity Hew York City. More in general we can view a KG under a dual perspective: as a directedlabeled mti-graph where nodes represent entities or literals and labeled edges repre- sent specific relationships between entities or between an entity and a literal and as a set of statems also refered to as facts having the fom of subject-redicate-bjecttrle e.g. (New York City country United States) and (Mev York City settled 1624). In the following we will use the notation (h t) (head relation tail) to identifya statement in KG as frequent in the literature about KG embeddings.
The entities described in KGs are monly organized using a set of spes e.g-City and Country also referred to as concepts classes or data types (when referred
to literals). For example the statement (eu York City type City) states that the entity Mew York City has type City. Indeed this types are often defined in what isgenerally refered to as the ontology [21]. An ontology is a formal specification of themeaning of types and relationships expressed as a set of logical constraints and rules qqpmo ]dxuspe oddsupon information extracted from Wikipedia describes more than 4 million entities andhas 3 billion statements' .
While KGs can be described using a graph a nice and simple way to visualize aknowledge graph is considering it as a 3-order adjacency tensor (i.e. a 3-dimensional tensor describing the structure of the KG). Formally a 3-dimensional adjacency tensoris definedasTRNRN where Nis the number ofentities andRis thenumber ofrelationships. Each dimension of the tensor corresponds to (head relation tai1) respectively.
More formally assume we have a KG 9 = {(e r) e) × S x where 6 and denote the sets of entities and relations in the KG respectively with || = N and [|= R. The adjacency tensor T R/xR×N is defined as follows:
To visualize this imagine a simple adjacency matrix that represents a single relation such as the country relation: the two dimensions of the matrix correspond to the headentity and the tail entity. Each entity corresponds to an unique index: given a triple (MewYork City country United States) we have a 1 in the cell of the matrix corre- sponding to the intersection between the i-th row and the j-th column where i. j Nare the indices associated with Hew York City and United States respectively. Oncontains a 0. If we consider more than one relationship and we stack them togethe we the other hand any cell in the adjacency matrx corresponding to triple not in the KGof a KG. See Figure 1 for a simple visualization of this concept. obtain a 3dimensional tensor generally refered to as the binary tensor reresentatin
Figure 2. Starting from a knowledge graph embedding methods generate representations of the elements of thkwlahthtdirameettVectors ecode latent propeties of the graph ad frxample similar ctities tend to be described with siilar vectors.
The term “knowledge graph embeddings"” refers to the generation of vector repre-sentations of thelements that form akowledge graphEentially what most methods do is to create a vector for each entity and each relation; these embeddings are gener-ated in such a way to capture latent properties of the semantics in the knowledge graph:similar entities and similar relationships will be represented with similar vectors. Fig- ure 2 provides an intuitive example of what a knowledge graph embedding method does.The tensor representation introduced above is frequently used in many KG embedding methods that learm embeddings by using dimensionality reduction techniques over thetensor.
(with values ranging from 100 dimensions to 1000 dimensions) and one key aspect is The elements are generally represented in a vector space with low dimensionalityuse of vector similarity measures (e.g. cosine similarity in which two vectors are more qm pad aq ue eus aeds nn e u :ues jo uoou o q uaisimilar if the angle between them is small).
tween entities. This task is generally refered to as link prediction or knowledge graph -q sdqsuean mou uppe s puxo on sem pug on s!yse ueodu upletion. Adding new facts can be done with the use of logical inference. For ex-ample from a triple (Washington D.C. capital United States) we can infer (Mashington D.C. country United States). Inferring this last fact es frombackground knowledge encoded in an axiom that specify that if a city is a capital of a country it is also part of that country (e.g. as encoded by a first order logic rule suchas X Y : capital(X Y) > country(X Y). Unfortunately many knowledge graphs havemany observed facts and fewer axioms or rules [87].
KG embeddings can be used for link prediction since they show interesting predic-tive abilities and are not directly constrained by logical rules. This property es at the
cost of not being directly interpretable (i.e. the vector representations now encode thelatent meaning of the entity/relationship). The explainability of this prediction is often difficult because the result es from the bination of latent factors that are embed-ded in a vector space and an evaluation of the inductive abilities of these methods is stillan open problem [87].
Knowledge graph ebeddings projected inthe vector space tend to show interestinglatent properties [61]; for example similar entities tend to be close in the vector space. The value of similarity in the latent space is a function that depends on the way knowl-edge graph embeddings are generated. Similarity is also important under the point ofview of explaining the meaning. For instance we might not know the meaning of the entity Nev York City but it can be infered from its topic by looking at closest entitiesin the geometric space (ie. Washington D.C. and United States).
The ponents of the vectors representing the entities and relations are not ex-plainable themselves and it can be hard to assign a natural language label that describesthe meaning of that ponent. Howeve we can observe how different entities and rela- tionships are related within the graph by analyzing its structure which was also used togenerate the vector-based representations. In addition the training is driven by a similar- ity principle which can be easily understood. For example similar entities have similarembedding representations and the same is true for similar relationships. Thus while itu x sn s is not possible to explain the exact difference between two vectors of two entities weuse these vectors and the additional information to enrich the network capabilities.
Knowledge graph embeddings have been used in different contexts including rec-ommendation [40 91 106] visual relationship detection [4] and knowledge base -knowledge inside deep neural networks thus enriching the explainability of pure black- pletion [11]. Moreover knowledge graph embeddings can be used to integrate semanticbox neural networks [48 38] but they also e with some limitations.
graphs and how to evaluate them. We discuss related work of the field by mentioningthe approaches that improved the state-of-the-art results. Then we focus on knowledge sppqdespoumoqqed oddns o supqbe adopted to provide explanations by describing the relevant state-of-the-art approaches. Similarity es has a key factor also in the context of explainability in remendersystems for example similarity is a key notion o expressuggestions to user.
1.1. Overview of this Chapter
This chapter provides an overview of the fieldin which we describe how KG embeddingsare generated and which are the most inffuential approaches in the filed up to dateMoreover the chapter should also describe which are the possible usages for KG embeddings in the context of explainability.In the recent literature many approaches for knowledegraph embeddings have been proposed; we summarize the most relevant models by fo-o uo oed pue seap ax a uo Susn
In Section 2 we give a more detailed overview related to how a knowledge graphmost popular models and then we will briefly explain how information that does not embedding method can be defined and trained. We will describe TransE [11] one of thee from the knowledge graph can be used to extend the capabilities of the embedding
models. This willbe a general introduction that should help the reader understand howthe methods introduced in the other sections work.
In Section 3 we describe the approaches we have selected. We summarize what re-searchers have experimented within the field giving to the reader the possibility of ex-it is difficult to describe which is the best model for a specific task becaue evaluation ploring different posible ways of generating knowledge graph embeddings. Note thatresults are greatly influenced by hyper-parameters (see Section 3.5). Nevertheless we think that most of the approaches have laid the basis for further development in the fieldand are thus worth describing. We then describe how knowledge graph embeddings areevaluated showing that the main task is link prediction and that the datasets used have changed over the years. Link prediction is a task that requires high explainability some-ComplEx [88] is often considered as one of the best performing models [4] and gives thing that in the context of knowledge graph embeddings is often missing. In general stable results in inductive reasoning tasks [87].
Then in Section 4 we focus on explainability. Explainability is a dificult term toare sub-symbolic representations of entities in which latent factors are encoded. Knowl edge graph embeddings can be used for link prediction but the prediction is the resultof the bination of latent factors that are not directly interpretable. However there is recent literature that explores the usage of embeddings in the context of explainable andlogical inferences.
and we describe possible future directions for the field. We conclude this chapter in Section 5 where we summarize our main conclusions
Adidirional ResourcesSeveral works that provide an overview of knowledge graph emnicely writen survey of aproaches that are meant to support the embedding of knowl dodqedge graph literals and to [92] for another overview on knowledge graph embeddings.po jo suoeusada ooqus-qns paod supoqua qdea pmou sy there is a recent increasing interest in finding ways to interpret how these representationsinteract [1]. Inductive capabilities of knowledge graph embeddings methods have been recently evaluated [87].
2. Knowledge Graph Embeddings
A Short Primer In this first part we are going to define the general elements that char-acterize aknowledge graph embedding method To betterillustratehow knowledge graphembeddings are created we focus our explanation on one of the seminal approaches of the field TransE [11]. We will introduce how TransE embeddings can be generated andhow a method like TransE can be extended to consider information that is not included in the set of triples. While we will describe TransE-specific concepts most of what it isexplained in this section is still valid for other methods in the state of the art.
exists [11 67 96 52 88]. In 2011 RESCAL [67] was the first infuential model to cre- Nowadays a plethora of approaches to generate embedded representations of KGstensor factorization approach upon the 3-dimensional tensor generated by considering ate embedded representations of entities and relationships from a KG by relying on asubject entity predicate entity and object entity as the 3 dimensions of the tensor. There
