scoping

Scoping, Linking and Indexing

Jan Köhnlein / Moritz Eysholdt – itemis AG

Agenda

I. Overview

II. Scope Usage

III. Scope Implementation

IV. The Index

V. Best Practices

I. Overview

EMF: Containment vs Cross References

Containment references form treesCross references extend trees to graphs

Cross References in DSLs

:PlacesModel

Hamburg:Place Berlin:Place Sven:Person

place Hamburgplace Berlin!person Sven visited Hamburg

PlacesModel: (persons+=Person | places+=Place)*;!Place: "place" name=ID ("{" (persons+=Person | places+=Place)* "}")?;!Person: "person" name=ID "visited" visited=[Place|ID];

References in Xtext Grammars


Containment References in Xtext Grammars

containment reference

rule


Cross References in Xtext Grammars

cross reference

target EClass identifier syntax

Cross References in Xtext Editors

Hyperlink Validation

ContentAssist Hover

FindReferences

II. Scope Usage

Scopes: Xtext’s abstraction to relate names and model elements

Linking ContentAssist

Serializing

If you understand how Xtext uses scopes, you will know how to implement your scopes.

The Three Use Cases:

Scopes: The Three Use Cases:Linking ContentAssist Serializing

Linking


place Hamburg!person Sven visited Hamburg

Person: "person" name=ID "visited" visited=[Place|ID];

IScope scope = scopeProvider.getScope(context, reference);

IEObjectDescription element = scope.getSingleElement(name);

EObject target = EcoreUtil.resolve(element.getEObjectOrProxy(), context);

if (type.isInstance(target)) {

context.eSet(reference, target);

}

Parsed Document Grammar

Xtext’s linking implementation




For context (Person “Sven”) find target EObject of cross reference “visited”. Name must be “Hamburg”. Type must be “Place”


IEObjectDescription element = scope.getSingleElement(name);

EObject target = element.getEObjectOrProxy();

if (type.isInstance(target)) {

context.eSet(reference, target);

}


ContentAssist




For context (Person “Sven”) and cross reference “visited”, find the names of all valid target elements.


Iterable<IEObjectDescription> elements = scope.getAllElements();

List<QualifiedName> names = new ArrayList<>();

for(IEObjectDescription desc : elements) {

names.add(desc.getName());

}


Serializing


Serializing

XtextResource

ModelModelTextual!Model

AST

Parser

Serializer

load()

save()




IEObjectDescription element = scope.getSingleElement(target);

QualifiedName name = element.getName();

For context (Person “Sven”), cross reference “visited” and target “Hamburg” find the target’s name.

model:EObject

name=SvenSven:EObject

name=HamburgHamburg:EObject

IScope scope = scopeProvider.getScope(context, reference);!!!scope.getSingleElement(name);!!scope.getAllElements();!!scope.getSingleElement(target);

Scopes: The Three Use Cases

Linking

ContentAssist

Serializing

IEObjectDescription

public interface IEObjectDescription { QualifiedName getName();! EObject getEObjectOrProxy(); EClass getEClass(); String getUserData(String key); String[] getUserDataKeys(); ! (…) }

• Handle for model element to avoid loading

• Name and type for model element

• User data

• Can be persisted in index

• Multiple descriptions for same EObject

Utils

QualifiedName name = QualifiedName.create("org", "eclipse", "xtext"); IEObjectDescription description = EObjectDescription.create(name, element); IScope scope1 = Scopes.scopeFor(eObjectList);IScope scope2 = new SimpleScope(eObjectDescriptionList);

III. Scope Implementation

Default Scoping Configuration

fragment = scoping.ImportNamespacesScopingFragment {}fragment = exporting.QualifiedNamesFragment {}

• Uses value of attribute “name“ • References use qualified names • Import statements for qualified names • Name shadowing • Cross-file references using index • Honors project dependencies

Inside a Scope

place Hamburg

place Berlin

place Kiel

!person Sven visited Hamburg

HamburgBerlinKiel

getScope(personSven, referenceVisited)

<returns>

Inside a Scope: Nesting

place Germany { person Sven visited Brazil place Kiel}!place Brazil

Kiel


<returns>

GermanyGermany.Kiel

Brazil

<parent>

Delegate to parent scope if name not found.



Kiel


<returns>

GermanyGermany.Kiel

Brazil

<parent>



Kiel


<returns>

GermanyGermany.Kiel

Brazil

<parent>

Delegate to parent scope if name not found.

Inside a Scope: Name Shadowing

place Kiel { person Sven visited Brazil place Brazil}!place Brazil

Brazil


<returns>

BrazilKiel

Kiel.Brazil

<parent>

Qualified Names

• Qualified Names consist of one or more segments

• Qualified Names are expected to be unique in a specific context

• Bind IQualifiedNameProvider in your RuntimeModule

• Default: Collect values of name attributes of all containers

fragment = exporting.QualifiedNamesFragment {}

Qualified Names in the Grammar

Person: "person" name=ID "visited" // visited=[Place] // bad, defaults to ID // visited=[Place|ID] // bad, ID does not allow “.” visited=[Place|FQN]; // good!FQN: ID ("." ID)*;

• If you extend Xbase.xtext or Xtype.xtext, they have a rule for QualifiedNames • If you defined your own, you may use org.eclipse.xtext.conversion.impl.QualifiedNameValueConverter

Inside a Scope: Name Qualification

place Germany { place Kiel { person Sven visited France.Paris place Brazil } place Hamburg}place France { place Paris}

BrazilgetScope(personSven, referenceVisited)

<returns>

KielKiel.BrazilHamburg

<parent>

GermanyGermany.KielGermany.Kiel.BrazilGermany.HamburgFranceFrance.Paris

<parent>

place Germany { place Kiel { person Sven visited France.Paris place Brazil } place Hamburg}place France { place Paris}

Inside a Scope: Name Qualification

BrazilgetScope(personSven, referenceVisited)

<returns>

KielKiel.BrazilHamburg

<parent>

GermanyGermany.KielGermany.Kiel.BrazilGermany.HamburgFranceFrance.Paris

<parent>

Default Implementation for Imports

fragment = scoping.ImportNamespacesScopingFragment {}

Import: "import" importedNamespace=FQNWithWildcard;!FQNWithWildcard: FQN ("." "*")?;

MyLanguageScopeProvider

ImportedNamespaceAwareLocalScopeProvider

Default Implementation for Cross-File References

fragment = scoping.ImportNamespacesScopingFragment {}

MyLanguageScopeProvider

ImportedNamespaceAwareLocalScopeProvider

DefaultGlobalScopeProvider

Index

Cross-File References

place SouthAmerica { place Brazil }place USA { place California }

west.places

import USA.*import SouthAmerica.Brazil!place Kiel { person Jan visited California person Sven visited Brazil person Anton visited SouthAmerica.Brazil place Brazil}place Luenen { person Alex visited Brazil }

home.places

• Import via FQN

• Import via wildcard

• Cross-file without import

• Imports are filename agnostic

• Local names have precedence



west.places


home.places Q: Why did Alex not visit Brazil close to Kiel?



west.places


home.places Q: Why did Alex not visit Brazil close to Kiel?

A: For context “Lünen” it has the name “Kiel.Brazil”

Implementing own Scopes (extends AbstractDeclarativeScopeProvider)

public class PlacesScopeProvider extends AbstractDeclarativeScopeProvider {! // method is called reflectively via name convention public IScope scope_Person_visited(Person person, EReference reference) { PlacesModel model = EcoreUtil2.getContainerOfType(person, PlacesModel.class); List<Place> allPlaces = EcoreUtil2.getAllContentsOfType(model, Place.class); return Scopes.scopeFor(allPlaces); }}

• need unit tests to guarantee method is called • java.reflect cumbersome during debugging • first parameter (context) may be less specific if parser is in lookahead • don’t forget to register an exception handler, otherwise exceptions are ignored

Implementing own Scopes (implements IScopeProvider)public class PlacesScopeProvider implements IScopeProvider {! @Inject // obtain the next ScopeProvider in the hierarchy @Named(AbstractDeclarativeScopeProvider.NAMED_DELEGATE) private IScopeProvider delegate;! // match references and dispatch to specific implementation public IScope getScope(EObject context, EReference reference) { if (reference == PlacesPackage.Literals.PERSON__VISITED) { PlacesModel model = EcoreUtil2.getContainerOfType(context, PlacesModel.class); return getPersonVisitedScope(model); } return delegate.getScope(context, reference); }! // create scope for PERSON__VISITED private IScope getPersonVisitedScope(PlacesModel model) { // if we can't create a scope, NEVER return null but return IScope.NULLSCOPE instead if (model == null) return IScope.NULLSCOPE; // build some scope. // This one includes all elements from the same file by their simple names List<Place> allPlaces = EcoreUtil2.getAllContentsOfType(model, Place.class); return Scopes.scopeFor(allPlaces); }}

IV. The Index

Use cases for the Index

• Filename-agnostic resolution of names

• Validation of cross-references without loading targets

• Computation of affected files during incremental build

• Reverse-reference lookup

Interactions with the Index

Index

Builder

Scoping

<updates, reads>

<reads>

<reads>

<reads>

Data Structure of the Index (simplified)

ResourceDescriptions

uri : URIimportedNames : List<String>

ResourceDescription

sourceEObjectURI : URItargetEObjectURI : URI

ReferenceDescriptionname : StringeObjectFragment: StringuserData : Map<String, String>

EObjectDescription

[0..*][0..*]

[0..*]

Index Population

incremental Build

resource change <triggers>

1. Index Exported Names

2. Resolve References and Validate

Index Population

incremental Build

resource change <triggers>

During computation of exported names, you CANNOT resolve references.

1. Index Exported Names

2. Resolve References and Validate

Customize Indexing of an Xtext-Language #1

public Class<? extends IQualifiedNameProvider> bindIQualifiedNameProvider() { return MyQualifiedNameProvider.class;}

MyLanguageRuntimeModule

public interface IQualifiedNameProvider extends Function<EObject, QualifiedName> {! /** * @return the qualified name for the given object, * <code>null</code> if this {@link IQualifiedNameProvider} is not * responsible or if the given object doesn't have qualified name. */ QualifiedName getFullyQualifiedName(EObject obj);}

Customize Indexing of an Xtext-Language #2@ImplementedBy(DefaultResourceDescriptionStrategy.class)public interface IDefaultResourceDescriptionStrategy {! /** * Calculates the {@link IEObjectDescription}s for * <code>eObject</code> and passes them to the acceptor. * * @return true if the children of <code>eObject</code> should be traversed. */ boolean createEObjectDescriptions( EObject eObject, IAcceptor<IEObjectDescription> acceptor );! /** … */ boolean createReferenceDescriptions( EObject eObject, URI exportedContainerURI, IAcceptor<IReferenceDescription> acceptor );}

V. Best Practices

Best Practices

• Do not scope to restrictively…

• …rather adapt validation and content assist.

• Avoid resolving elements when filtering scopes.

• Names should not depend on cross-references.

That’s all, folks!

scoping

Software

person places

place sven

placeidfor context person

use cases

graphscross references

placeidcross references

placeidcontainment references

placeiscope scope