http push api guide

exalead T M

EXALEAD HTTP/REST PUSH API GUIDE

AN EXALEAD S.A. OEM SUPPORT DOCUMENT

This document details the Exalead HTTP-Push API

Doc. No. EN.120.002.1-V4.6.1 - March 31, 2008

Copyright © 2008 by Exalead S.A. All rights reserved.

exalead one:enterprise HTTP/REST PUSH API GUIDE

Legal Notice

Information in this document is subject to change without notice and does not represent a commitment on the part of Exalead S.A..

This document is Copyright © by Exalead S.A. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying or recording, for any purpose without the express written permission of Exalead S.A..

Exalead, ExaScript, CloudView and its associated logos are Registered Trademarks of Exalead S.A..

This document makes reference to other names and products that are Trademarks of their respective owners.

The software described in this document is Copyright © by Exalead S.A. and is supplied under a licence agreement, a nondisclosure agreement, or both. It is against the law to copy or transmit this software by any medium except as specifically sanctioned by these agreements.

For more information about Exalead:

• See www.exalead.com

• Write to us at:

Exalead 10 place de la Madeleine 75008 Paris FRANCE

• Telephone: +33 (0)1 55 35 26 26• Fax: +33 (0)1 55 35 26 27

For Product Support:

http://www.exalead.com/software/services/support/

US and Canada only:

• Hotline: (888) 247-9214 (Monday-Friday, 9am - 9pm ET)

• Email: [email protected] (General Inquiries) [email protected] (OEM-Related Inquiries)

Outside the US:

• Hotline: +33 1 55 35 26 00

• Email: [email protected]

Exalead S.A. is incorporated with a nominal capital of 113 866.80 €. RCS Paris B 432 887 875 APE 722C

This product has several patents.

DOC. NO. EN.120.002.1-V4.6.1 - MARCH 31, 2008 2© 2008 BY EXALEAD S.A. REPRODUCTION AND DISCLOSURE PROHIBITED.

http://www.exalead.com/

mailto:[email protected]

mailto:[email protected]

exalead one:enterprise HTTP/REST PUSH API GUIDE TABLE OF CONTENTS


Table of Contents

Preface

About this Guide . . . . . . . . . . . . . . . . . . . . . . . . . . 5Acronyms and Terminology . . . . . . . . . . . . . . . . . . . . . . 7

Section 1 API Conventions and methods

API Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . 9POST add_document . . . . . . . . . . . . . . . 11POST add_document_list . . . . . . . . . . . . . . 12POST delete_document . . . . . . . . . . . . . . 13POST delete_document_list . . . . . . . . . . . . . 14POST delete_document_collection . . . . . . . . . . . . 15GET get_document_status . . . . . . . . . . . . . . 16GET get_document_status_list . . . . . . . . . . . . . 17POST set_checkpoint . . . . . . . . . . . . . . . 18GET get_checkpoint . . . . . . . . . . . . . . . 19POST clear_all_checkpoints . . . . . . . . . . . . . 20POST open_document_status_collection_iterator . . . . . . . . 21POST next_document_status_collection_iterator . . . . . . . . . 22POST next_batch_document_status_collection_iterator . . . . . . . 23POST close_document_status_collection_iterator . . . . . . . . 24POST open_checkpoint_iterator . . . . . . . . . . . . 25POST next_checkpoint_iterator . . . . . . . . . . . . 26POST next_batch_checkpoint_iterator . . . . . . . . . . . 27POST close_checkpoint_iterator . . . . . . . . . . . . 28POST create_partial_flush_task . . . . . . . . . . . . 29POST get_task_status . . . . . . . . . . . . . . 30

exalead T M

PREFACE

How to use this guide

exalead one:enterprise PREFACE

HTTP/REST PUSH API GUIDE ABOUT THIS GUIDE

About this Guide

Revision history

Documentation set The exalead one:enterprise documentation set is as follows.The exalead one:enterprise documentation set is as follows. The technical and user reference guides are:

The programming API guides are:

The technology guides are:

Document audience This document has been written to explain how to use the exalead one:enterprise Push API. It is for people who have experience with:

• HTTP API

Ver. Date Author Document revision history

1.0 May 31, 2007 A. Derbel Creation of document.

4.6.0 March 4, 2008 A. Derbel New versioning and format for new exalead one:enterprise v4.6 release.

4.6.1 March 31, 2008 A. Derbel Updates for version 2 of the Push API for exalead one:enterprise v4.6.

Document number Title

EN.120.001 Exalead one:enterprise Administration

EN.120.002 Exalead one:enterprise Connectors

EN.120.003 Exalead one:enterprise Security

EN.120.004 Exalead one:enterprise HTML Front-end

EN.120.007 Exalead one:enterprise User

EN.120.008 Exalead one:enterprise Installation


EN.120.002.1 Exalead one:enterprise HTTP/REST Push API

EN.120.002.2 Exalead one:enterprise Push API

EN.120.018 Exalead one:enterprise Search Front-end API: XML v10

EN.120.020 Exalead one:enterprise Search Front-end API: XML v3.1

EN.120.021 Exalead one:enterprise Query Language


EN.120.010 Exalead one:enterprise Categorization

EN.120.011 Exalead one:enterprise Filtering



HTTP/REST PUSH API GUIDE ABOUT THIS GUIDE

Purpose and application scope

The Exalead PUSH API is the API available for Exalead partners and contractors to index a new data source from the exalead one:enterprise product.

The PUSH API is natively a simple HTTP/Rest API. The API user may either use this Rest API directly from the language of its choice, or use provided client-side wrappers for the API (C#, Java, Ruby, Perl, ExaScript).

This document describes version 2 of the API at the HTTP/Rest level, including:

– HTTP url syntax,

– parameters convention,

– types serialization/de-serialization,

– error handling and

– security aspects.

Please refer to the Push API documentation for more details about global notions.

API users should refer to the general Push API documentation of their client-side wrappers documentation for more detailed explanations and examples.

Formatting conventions

The following formatting conventions are used in this document.

The text style Is used for

Italic Titles used in cross-references and variable names.

Output font

All output and for anything you would type in a console window.

COURRIER NEW FONT Text labels from a graphic user interface.



HTTP/REST PUSH API GUIDE ACRONYMS AND TERMINOLOGY

Acronyms and Terminology

Acronyms The following acronyms are used in this guide.

Terminology Some important terms used in this document are defined below.

Acronym Means

HTTP Hypertext Transfer Protocol

URI Universal Resource Identifier

PAPI Push API

IP Internet Protocol

URL Universal Resource Locator

XML Extensible Markup Language

Term Definition

Command A "Command" refers to the function requested on the server (add_document, delete_document, ...)

URL The "URL" refers to the address on which the client must send POST or GET request. It includes the HTTP Push Server host and port number, the conventional prefix for the connectors command, the source name and, lastly, the Command.

For example: http://myexaserver:10011/private/connectors/mysource/add_document


exalead T M

SECTION 1

API CONVENTIONS AND METHODS

A description of the HTTP/Rest Push API conventions and methods

exalead one:enterprise SECTION 1 API CONVENTIONS AND METHODS

HTTP/REST PUSH API GUIDE API CONVENTIONS

API Conventions

Command parameters

The parameters can be sent in different ways using the:

The recommended way parameters should be sent to the server is specified for each parameter in the Command description: [URL] or [FORM].

HTTP Methods The HTTP methods used are the following:

HTTP Header Parameters

To detect hazardous mixes of the two API versions between the client and the server side, an extra Parameter is added to every request.

Command response

The Exalead PUSH API operations processing may be asynchronous. This means that requested add or delete operations are accepted but we don't know for sure when they will be performed. Subsequent errors can be retrieved asynchronously, by retrieving the status of Document.

But some errors can occurs at a lower level, here is the presentation of the general default HTTP responses; special cases are described below for the Commands concerned.

Push API, only specified methods are authorized for each Command.

Parameter Description

URL The parameter in the request URL, for example: ...../addDocument?uri='file://mydir/file1.doc'

FORM the parameter is part of the form data


GET The GET method.

POST The POST method can be encoded as MIME multipart/form-data content-type (RFC 2388), or application/x-www-form-urlencoded


Papi-Version The possible values for the PAPI version (at this moment) are the following:

• PAPI_v1.0

• PAPI_v2.0

HTTP Response Description

OK No problem during parameters de-serialization process.

BAD_REQUEST An error occurred while parameters parsing or during command treatment. The body of the result contains the error description.

METHOD_NOT_ALLOWED Usage of POST or GET method is strict with the HTTP



HTTP/REST PUSH API GUIDE API CONVENTIONS

Example 1 - Convention for error serialization

List of possible error types:

• InvalidConnectorError

• InvalidParameterError

• InvalidFilterError

<error><type> ... </type><short_message> ... </short_message><message> ... </message>

</error>



HTTP/REST PUSH API GUIDE POST add_document

POST add_document

Parameters The parameters are described in the table below.

Description Request for a document addition.

For better performance, it's recommended to use a multipart/form-data instead of application/x-www-form-urlencoded.

Response (default)


PAPI_uri [URL] The URI parameter is directly the string of the document's URI.

PAPI_stamp [FORM] (optional) The stamp parameter is the string representing the document's Stamp.

PAPI_meta_<metaname> [FORM] The meta_* parameter is a string containing the value of the metadata referenced by metaname.

A metadata does not have to be unique, but you have to check for which of them it make senses to have multiple values on the metadata description page (refer to Exalead Push API document).

PAPI_part_bytes:<name> [FORM] The part_bytes parameter is the content of the document's part that is identified by 'name'.

PAPI_part_ext:<name> [FORM] (optional)

The part_ext parameter is the extension hint of the document's part that is identified by 'name'.

PAPI_part_mime:<name> [FORM] (optional)

The part_mime parameter is the mime hint of the document's part that is identified by 'name'.



HTTP/REST PUSH API GUIDE POST add_document_list

POST add_document_list


NOTE These parameters must be repeated (with a different id) for very document you want to send.For better performance, it's recommended to use a multipart/form-data instead of application/x-www-form-urlencoded.

Description Request for a document addition on the collection sent.

For better performance, it's recommended to use a multipart/form-data instead of application/x-www-form-urlencoded.

Response (default)


PAPI_<id>:uri [FORM] The URI parameter is directly the string of the document's URI.

PAPI_<id>:stamp (optional) [FORM]

The stamp parameter is the string representing the document's Stamp.

PAPI_<id>:meta_<metaname>[FORM]

The meta_* parameter is a string containing the value of the metadata referenced by metaname.

PAPI_<id>:part_bytes:<name> [FORM]

The part_bytes parameter is the content of the document's part that is identified by 'name'.

PAPI_<id>:part_ext:<name> [FORM] (optional)

The part_ext parameter is the extension hint of the document's part that is identified by 'name'.

PAPI_<id>:part_mime:<name> [FORM] (optional)

The part_mime parameter is the mime hint of the document's part that is identified by 'name'.



HTTP/REST PUSH API GUIDE POST delete_document

POST delete_document


Description Request for a document deletion.

Response (default)


PAPI_uri [URL] The URI parameter is directly the string of the document's uri.



HTTP/REST PUSH API GUIDE POST delete_document_list

POST delete_document_list


Description Request for a document deletion on the specified URI list.

Response (default)


PAPI_uriList [FORM] The uriList parameter is directly the string of the document's uriList serialized using the following xml serialization format:

<uriList>

<uri>file://....</uri>

<uri>file://....</uri>

</uriList>



HTTP/REST PUSH API GUIDE POST delete_document_collection

POST delete_document_collection


Description Request for a document deletion on a collection selected by the filter.

Response An error status indicating an InvalidFilterError occurred can be returned in case the filter syntax is incorrect.


PAPI_filter [URL] The filter parameter is directly the string representation of the filter.

For more details, see the filter chapter in the Exalead one:enterprise Push API Guide.



HTTP/REST PUSH API GUIDE GET get_document_status

GET get_document_status


Description Retrieve from the Indexing System the status of the document specified by the URI parameter. The structure is serialized and returned in the response body.

Response If successful (status = OK), then the body contains the serialized form of the DocumentStatus in XML format:

Where status can take the following values: EXISTS, MISSING, ERROR


PAPI_uri [URL] The URI parameter is directly the string of the document's URI.

<DocumentStatus><uri> ... </uri><stamp> ... </stamp><status> ... </status><message> ... </message>

</DocumentStatus>



HTTP/REST PUSH API GUIDE GET get_document_status_list

GET get_document_status_list


Description Retrieve from the Indexing System the status of all documents specified by the uriList parameter. The structure is serialized and returned in the response body.

Response If successful (status = OK), then the body contains the serialized form of the DocumentStatus[] in XML format:


PAPI_uriList [FORM] The uriList parameter is directly the string of the document's uriList serialized using the following XML format:<uriList> <uri>file://....</uri> <uri>file://....</uri></uriList>

<DocumentStatusList><DocumentStatus>

<uri> ... </uri><stamp> ... </stamp><status> ... </status><message> ... </message>

</DocumentStatus>...

</DocumentStatusList>



HTTP/REST PUSH API GUIDE POST set_checkpoint

POST set_checkpoint


Description Set the given checkpoint. If the optional name is given, then the concerned checkpoint is changed.

Response (default)


PAPI_checkpoint [URL] The checkpoint parameter is directly the string of the checkpoint's value.

PAPI_name(optional) [URL]

This optional parameter can be used in case you need to manage many checkpoints for a connector.



HTTP/REST PUSH API GUIDE GET get_checkpoint

GET get_checkpoint


Description Get the last saved checkpoint. If the optional name is given, then the concerned checkpoint is returned.

Response If successful (status = OK), then the body contains the serialized form of the checkpoint, which is directly the string value of the checkpoint


PAPI_name(optional) [URL]

This optional parameter can be used in case you need to manage many checkpoints for a connector.



HTTP/REST PUSH API GUIDE POST clear_all_checkpoints

POST clear_all_checkpoints

Description Delete all checkpoints created using an optional name.

Response (default)



HTTP/REST PUSH API GUIDE POST open_document_status_collection_iterator

POST open_document_status_collection_iterator


Description Open an iterator on a document collection matching the filter given as parameter.

Response If successful (status = OK), then the body contains the serialized form of the IteratorID, which is directly the string value of the IteratorID (int).

The IteratorID will be used to address the correct iterator (in case of multiple concurrent iterator) stored in the server context.

An error status indicating an InvalidFilterError occurred can be returned in case the filter syntax is incorrect.


PAPI_filter [URL] The filter parameter is directly the string representation of the filter.



HTTP/REST PUSH API GUIDE POST next_document_status_collection_iterator

POST next_document_status_collection_iterator


Description Return the next DocumentStatus available in the enumeration associated to the iteratorID.

Response If successful (status = OK), then the body contains the serialized form of the DocumentStatus in XML format:

Where status can take the following values: EXISTS, MISSING, ERROR

If the end of the iterator has been reached, then <null/> is returned.

An error status indicating an InvalidIteratorError occurred can be returned in case the iteratorID is invalid.


PAPI_iteratorID [URL] The iteratorID parameter is directly the string representation of the integer value.

<DocumentStatus><uri> ... </uri><stamp> ... </stamp><status> ... </status><message> ... </message>

</DocumentStatus>



HTTP/REST PUSH API GUIDE POST next_batch_document_status_collection_iterator

POST next_batch_document_status_collection_iterator


Description Return the next DocumentStatus[] available in the enumeration associated to the iteratorID.

Response If successful (status = OK), then the body contains the serialized form of the DocumentStatus[] in XML format:

If no more results are available, then <DocumentStatusList/> is returned.



PAPI_iteratorID [URL]

The iteratorID parameter is directly the string representation of the integer value.

PAPI_count [URL] The count parameter is directly the string representation of the integer value.

<DocumentStatusList><DocumentStatus>

<uri> ... </uri><stamp> ... </stamp><status> ... </status><message> ... </message>

</DocumentStatus>...

</DocumentStatusList>



HTTP/REST PUSH API GUIDE POST close_document_status_collection_iterator

POST close_document_status_collection_iterator


Description Release resources allocated on the server side associated to the iteratorID.

Response (default)


PAPI_iteratorID [URL]

The iteratorID parameter is directly the string representation of the integer value.



HTTP/REST PUSH API GUIDE POST open_checkpoint_iterator

POST open_checkpoint_iterator


Description Open an iterator on the list of optional name used to create checkpoints.

Response In case of success (status = OK), then the body contains the serialized form of the IteratorID, which is directly the string value of the IteratorID (int).

The IteratorID will be used to address the correct iterator (in case of multiple concurrent iterator) stored in the server context.


N/A



HTTP/REST PUSH API GUIDE POST next_checkpoint_iterator

POST next_checkpoint_iterator


Description Return the next checkpoint name available in the enumeration associated to the iteratorID.

Response In case of success (status = OK), then the body contains the serialized form of the Checkpoint name in XML format:

If the end of the iterator has been reached, then <null/> is returned.



PAPI_iteratorID[URL] The iteratorID parameter is directly the string representation of the integer value.

<Checkpoint> ... </Checkpoint>



HTTP/REST PUSH API GUIDE POST next_batch_checkpoint_iterator

POST next_batch_checkpoint_iterator


Description Return the next String[] available in the enumeration associated to the iteratorID.

Response In case of success (status = OK), then the body contains the serialized form of the String[] in XML format:

If no more results are available, then <CheckpointList/> is returned.




PAPI_count[URL] The count parameter is directly the string representation of the integer value.

<CheckpointList><Checkpoint> ... </Checkpoint>...

</CheckpointList>



HTTP/REST PUSH API GUIDE POST close_checkpoint_iterator

POST close_checkpoint_iterator


Description Release resources allocated on the server side associated to the iteratorID.

Response (default)





HTTP/REST PUSH API GUIDE POST create_partial_flush_task

POST create_partial_flush_task

Description Create on the server a partial flush (commit) operation asynchronously, and return a TaskID that could be used to monitor the status of the task.

Response In case of success, returns an integer, which is the TaskID of the created task.



HTTP/REST PUSH API GUIDE POST get_task_status

POST get_task_status


Description Check the status of a task previously created on the server .

Response The status of the task :

Status table:

1 : RUNNING

2 : FINISHED

3 : ERROR

In case of ERROR, the message field contains the description of the error.


PAPI_taskID[URL] The taskID parameter is directly the string representation of the integer value of the Task we want to monitor.

<TaskStatus><status>1,2 or 3</status><message>….</message>

</TaskStatus>


About Exalead

Founded in 2000 by search-engine pioneers, Exalead (www.exalead.com) is a global provider of software that is designed to simplify all aspects of information search and retrieval for organizations of all sizes. Based on the first and only unified technology platform for desktop, intranet or Web search, Exalead offers easier deployment, administration and use than any other enterprise-type search software. This is true whether for one or thousands of desktops, a small business or global enterprise, and conforms to any technology environment. It also adapts to user habits for a uniquely satisfying search experience.

Exalead software is used by leading banking and financial services, media, consumer packaged goods, research, retailing sports entertainment and telecommunications companies around the world, including Air Liquide, BNP Paribas and Carlson Wagonlit. Exalead is an operating unit of Qualis, an international holding company.

Call Exalead for Your Information Access Needs

Talk to us about your information access needs. Our sales technicians will be delighted to answer any questions you may have.

Exalead - France10 place de la Madeleine

75008 Paris, France

Tel: +33 (0)1 55 35 26 26Fax: +33 (0)1 55 35 26 27

Exalead - USA576 Folsom Street - 2nd floor

San Francisco, CA 94105

Tel: +1 (415) 230-3800Fax: +1 (415) 568-3375

Exalead - ItalyCorso Giuseppe Garibaldi, 86

20121 - MILANO, Italy

Tel: +39 02 62 71 10 10Fax: +39 02 62 71 10 11

Exalead - United KingdomInternational House

Stanley Bvd, HamiltonGlasgow G72 0BN

Tel: +44 (0)1698 404630Fax: +44 (0)1698 404639

Exalead - GermanyNiederlassung Deutschland

Robert-Bosch-Strasse 764293 Darmstadt

Tel: +49 6151 35 99 690-0Fax: +49 6151 35 99 690-35

[email protected] www.exalead.com

31 DOC. NO. EN.120.002.1-V4.6.1 - MARCH 31, 2008LAST PAGE

exalead T M

http push api guide

Documents