external master data in alfresco: integrating and keeping metadata consistent across multiple...

External Master Data in Alfresco:

Integrating and Keeping Metadata

Consistent Across Multiple Systems

Oksana Kurysheva @aviriel Alexey Vasyukov @avasyukov

Who are we?

Oksana Kurysheva

@aviriel Project Manager / Trainer / Queen of the bees

ITD Systems / Order of the Bee

Alexey Vasyukov

@avasyukov Chief Architect / Alvex Product Manager ITD Systems

Questions for the next 39 minutes

What is Master Data? 1

2

3

4

Storing Master Data in Alfresco

External Master Data

Complicated cases

5 Conclusion

What is Master Data? 1

2

3

4

Storing Master Data in Alfresco

External Master Data

Complicated cases

5 Conclusion

What is Master Data?

Master data management is an integration between Alfresco and a complex set of

third-party systems, that you are to meet when trying to use Alfresco in the wild.

What is Master Data? 1

2

3

4

Storing Master Data in Alfresco

Integrating Alfresco with external

Master Data

Complicated cases

Out of the box implementation

Data-list based implementation

5 Conclusion

3.1

3.2

Constraints in the content model

Constraints in the content model

Users can select a value from the list.

+ Master data changes quite frequently.

Business users do not like the idea to call IT every time they need new option in the drop-down list.

-

What is Master Data? 1

2 Storing Master Data in Alfresco

Out of the box implementation

Data-list based implementation

3.1

3.2

3

4

Integrating Alfresco with external

Master Data

Complicated cases

5 Conclusion

Data-list based implementation (1/5)

• Simple solution

• Several extensions to do it: • Tony Parzgnat implementation

https://github.com/parzgnat/alfresco-value-assistance

• Angel Borroy implementation

https://github.com/keensoft/alfresco-datalist-constraints

• Alvex releases older 2.1

<appearance>

<field id=“dm:color”>

<control template=“/form-controls/dynamic-dropdown.ftl”>

<control-param name=“pickListName”>Color</control-

param>

</control>

</field>

</appearance>

Data-list based implementation (2/5)

Data-list based implementation (3/5)

Solves basic issue quickly

• Business users can edit constraints in browser!

Data-list based implementation (4/5)

Updating a list of values

Performance

Permissions

Share-Tier

No native constraint

Still no sync with external Master Data

-

Data-list based implementation (5/5)

What is Master Data? 1

2

3

Storing Master Data in Alfresco

The Good

The Bad

3.1

3.2

4 Complicated cases

5 Conclusion

External Master Data

External Master Data

Assumptions:

• Master data is in 3rd party systems.

• These systems have remote APIs.

Let's query them for reference data.

Architectural Choice

Share Tier vs Repo Tier

Architectural Choice: Share-tier

Architectural Choice: Share-tier

<field id=“alvexdt:contract”>

<control template=“/alvex-masterData-select.ftl”>

<control-param name=“url”>

http://www.alvexsoftware.com/files/contacts_sample.json

</control-param>

<control-param name=“path”></control-param>

<control-param name=“label”>contactCompany</control-param>

<control-param name=“value”>contactCompany</control-param>

</control>

</field>

Architectural Choice: Repo Tier

Architectural Choice: Repo Tier

<constraint name="alvexdt:docStatus"

type="com.alvexcore.repo.masterdata.MasterDataConstraint">

<parameter name="dataSourceName">

<value>Test</value>

</parameter>

</constraint>

What is Master Data? 1

2 Storing Master Data in Alfresco

3

The Good

The Bad

3.1

3.2

4 Complicated cases

5 Conclusion

External Master Data

External Master Data

Quite obvious idea becomes tricky:

• Live data vs Cached data

• Performance and bottlenecks

• Changes in reference data

Dilemma: Live Data vs Cached Data

Always actual

Simple implementation

Slow network

Service is unavailable?

Live Data May be

outdated

Complex implementation

Fast

Always available

Cached Data

Performance

• Constraint is expected to be extremely fast

• Just reading field with list constraint with N values causes N+1 call to Constraint class.

• Caching of classifiers in Repo is not enough.

Changes in Reference Data

Update items

Processing Updates Leave old

value in existing fields

Migrate existing fields to the new one

Processing Deletion

Allow creating new data from Alfresco

Add new data

Implementation Details (1/5)

Implementation Details (2/5)

Implementation Details (3/5)

Implementation Details (4/5)

Implementation Details (5/5)

Changes in Reference Data

Update items

Processing Updates Leave old

value in existing fields

Migrate existing fields to the new one

Processing Deletion

Allow creating new data from Alfresco

Add new data

External Master Data: Demo

2

3

4

Storing Master Data in Alfresco

External Master Data

Complicated cases

4.1

4.2

Handling Tree-Like Structures

Constraints Inter-Dependencies

4.3 One-To-Many Relations

5 Conclusion

Complicated Cases

Reference data is complex IRL:

• Tree-like hierarchies

• Inter-dependencies of constraints

• One-to-many mapping (one master data

object to many content object properties)

Handing Tree-Like Structures

Handing Tree-Like Structures

• Jean Barmash implementation (2008)

https://forums.alfresco.com/comment/80963#co

mment-80963

Constraints Inter-Dependencies

One-To-Many Relations

What is Master Data? 1

2

3

4

Storing Master Data in Alfresco

External Master Data

Complicated cases

5 Conclusion

Fear and Loathing Under The Hood

Getting forms

Getting forms

Validations on Node Create/Edit

Validations on Node Create/Edit

Fear and Loathing Under The Hood: Summary

Overloaded components:

• /api/formdefinition

• MasterDataAwareScriptFormService

• MasterDataAwareIntegrityChecker

• MasterDataAwarePropertiesIntegrityEvent

Roadmap

• Completely solve Complicated Cases

• Extract Master Data from Alvex to a

separate extension

Resources

GitHub Repo:

https://github.com/ITDSystems/alvex

Contacts:

@aviriel & @avasyukov & @alvexsoftware

ping@itdhq.com