Customizing Microsoft FAST Search Server 2010 for SharePointKeenan Newton & Jeff FriedSenior Technical Product ManagersMicrosoft Corporation

SESSION CODE: OSP218

Do More With SearchAcross a Spectrum of Customization and Development

Configure Extend Create

Intranet SearchPeople Search

Site Search

Research PortalCase Management

Save Results to Excel file…..

IP Portfolio mgmtIntel/Surveillance

Drug Discovery….

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCreating Search VerticalsCustomizing FederationCustomizing Search Web PartsSearch Driven ApplicationsUsing FAST Query Language (FQL)

OSP217 What’s New in Enterprise Search in Microsoft SharePoint Server 2010

OSP206 Getting to know FAST Search Server 2010 for SharePoint

FAST Search Extends SharePoint Superset of capabilities; common base functionality and platform

End Users Content

Content search

People search

Common query and federation APIs

Common connectors and content APIs

Best High-End Content Search based on FAST ESP

Unique Social Search from SharePoint Server 2010

Common IT Pro and developer experience

FAST Search for SharePoint

SharePoint Server2010

Advanced Content Processing

+ Profiles

Search 2010 ArchitectureThe platform for Search Customization

SearchWeb Parts

SharePoint Search Index

OpenSearch /Custom Source

SharePoint Indexer

Federation OM Web Service, RSS

FASTSearch Index

FAST Indexer

What’s New in 2010?Primary Search Web Parts now UnsealedFederation now a key Public OM layer

All Web Parts built on federationQuery alteration, custom Runtimes, blending results from multiple sources

Web Service / RSS Enhancements

FAST Search / SharePoint Search:Shared Web Parts, RSS, Web ServiceShared Federation OMIndex and Crawling SeparateFAST unique Content Processing Pipeline

SearchWeb Parts

Federation / Query OM Web Service, RSS

Content Processing

Top Customization ScenariosModify the OOB End User Experience Add new Refinement category

Show results from federated location Modify the look and feel of OOB end user

experience Enable sorting by custom metadata Add visual Best Bet for upcoming sales event Configure different ranking for HR vs. Engineering

department

Create a new Search Verticals

Create new visual elements

Query & Result Pipeline Plug-ins

Query & Indexing Shims

Create new Search Driven Applications

Create new customer search experience Indexing content Define content for search Design search experience

Create new Audio/Video/Image search experience

Show Location refinement on Chart/Maps Show tags in tag cloud Enable export results to Spread Sheet Summarize Financial Information from customers

in Graphs Expand query terms based on synonyms defined

in Term Store Augment customer results with project

information Show popular customers/people inline with

search results Show people results from other sources

Show email results from personal mailbox on Exchange Server through the EWS

Index content from custom repositories like Documentum

Create content processing plug-ins to create new metadata

create a new customer page that shows: Customer Contact Details Customer Project Details Customer Contacts Internal Experts Customer related documents

Developer Extensibility

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCreating Search VerticalsCustomizing FederationCustomizing Search Web PartsSearch Driven ApplicationsUsing FAST Query Language (FQL)

Breaks down content to the smallest addressable chunks to build meaningUnderstands file encoding, data formats, and written languages Supports 400+ file formats, 80+ languages

Process content with advanced linguisticsAutomatic and detailed analysis creates a great search experience

FAST Content Processing Pipeline

Configure Optional Processing StepsXML Properties mapperOffensive Content FilterPipeline Extensibility

Calls external applications for custom item processing

Field CollapsingAdd Custom Processing

Content classificationGeo-taggingMachine translationSentiment Analysis

Extending Pipeline capabilitiesSafely add additional analysis and processing

Custom Entity Extraction

DEMO

Custom Entity Extraction SummaryCreated dictionary fileUsed PowerShell to copy dictionary file – Add-FASTSearchResourceActivated Entity Extractor in OptionalProcessing.xmlRestarted Document ProcessorsRe-crawled content

Pipeline Extensibility API

MotivationStraightforward way to add text analysis functionalityFlexibility and supportability

Example usesSentiment analysisTranslationAuto-Classification

MechanismJust before Mapper“any” binaryRuns in sandbox with timeout

…

ExtensibilityMapperStandard

processing

Pipeline Extensibility

DEMO

Pipeline Extensibility SummaryDetermine input and output crawled propertiesCreate executable that takes in 2 arguments.

Arg 1 - In file nameArg 2 – Out file nameDeploy to C:\FastSearch\bin

Modify pipelineextensibility.xml file – C:\FastSearch\etcSpecify executableSpecify input and output crawled properties

Create and map any crawled and managed propertiesRestart Document ProcessorsCrawl Full

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCreating Search VerticalsCustomizing FederationCustomizing Search Web PartsSearch-Driven ApplicationsUsing FAST Query Language (FQL)

“Out of the Box” Search ExperienceA powerful baseline for customization

Refinement summarizes and narrows resultsQuery federation brings together results(FAST) Previews provide rich interactionQuery Suggestions guide user keywords(FAST) User Context provides a personal experience

Search Vertical

DEMO

Search Vertical SummaryCreate Pages (search and results)Add TabsChange SearchBoxesRestarted Document ProcessorsRe-crawled content

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCreating Search VerticalsCustomizing FederationCustomizing Search Web PartsSearch-Driven ApplicationsUsing FAST Query Language (FQL)

Federation Extensibility ExamplesConfigure

New OpenSearch locationsConnect to Web Parts

ExtendCan’t do it since all the OOB run-times are sealed

CreateNew Federation Runtime (ILocationRuntime, IRefinableRuntime)

Federation Object Model

public sealed class Query Manager

public class LocationList

public class Location

public Interface ILocationRuntimeInterface

0..*

0..*

1

Responsible for managing all the locations

Responsible for aggregating results from all locations in this LocationList

Represents a federated location in the object model

Repository specific code for executing query and getting results

Custom Federation Source

DEMO

Creating a SQL Server Search Location

Create SqlServerLocationRuntime classInherits from ILocationRuntime and IRefinableMakes SQL requestCreates core results xmlPopulates Refiners

Create OSDX fileSpecifies configuration settingsUse ConnectionUrlTemplate element for connection string

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCustomizing FederationCustomizing Search Web PartsSearch-Driven ApplicationsUsing FAST Query Language (FQL)

Customizing Search Web Parts

Changing rendering behaviorOverride GetXPathNavigator,return an customized XPathNavigatorApplies to All Unsealed Search Web Parts

Changing data query or retrieval behaviorExtend the CoreResultsDataSource and CoreResultsDataSourceView or SearchResultsBaseDataSource and SearchResultsBaseDatasourceViewCan be used for creating a custom blending of data sources aka search locations

Visual Refinement Panel Web Part Summary

DEMO

Visual Refinement Panel Web Part Summary

Extend RefinementPanelWebPartOverride GetXPathNavigator

Call base.GetXPathNavigator to get refinement xmlCan read and manipulate xml before it is rendered

When using Charts use the Microsoft.Office.Server.WebControls.Chart class

Mixer Results Web Part

DEMO

Mixer Results Web Parts Summary

Extend DataSource and DatasourceView classesCreate child data sourcesOverride SetPropertiesOnQrda to set properties for the SharedQueryManagerConfigure locations for secondary data sources

Extend CoreResultsWebPartAdd code for adding additional sourcesOverride CreateDataSource and ConfigureDataSourceProperties,

create and configure data sourceOverride GetXPathNavigator, perform custom mixing here

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCreating Search VerticalsCustomizing FederationCustomizing Search Web PartsSearch-Driven ApplicationsUsing FAST Query Language (FQL)

Search-Driven Application demo

Demo

News and external opinion to monitor and send to

clients

Top information from Woodgrove…new market view report to send to clients

Drilldown to single view with all clues about a customer: portfolio, holdings, communications, annual and quarterly customer plans, etc…

Set of Customers to explore

Experts to help, with availability and rating

Immediate actions on selected items

View of information across different pivots, with drilldown

Finds customers beyond direct terms (REIT ~ real estate)

Start with a simple query, quickly find and explore

Content Crawling: bring in data from lots of placesOOB connectors to SharePoint (reports, account documents), exchange public folders, shared files; BDC with customization in SPD (no code) for customer portfolio/holdings

Content processing: creating metadataNames of holdings, offerings, key concepts, companies, peopleSynonyms for key concepts (real estate ~ REIT)

OOB web parts configured for style Federation, People Search, Search actions

Custom web parts for visual navigationRoll-up configured via results collapsingCustom relevance profileSharePoint workflows for act-on-selected-items

How would you create this?

Do More With SearchCreate new solutions powered by search

Solution examples:• Risk Management• IP Portfolio Mgmt• E-Discovery

• Build important applications quickly; innovate easily

• Combine Search with other SharePoint + Office capabilities

• Create New Relevance Profiles• Leverage Advanced Query and

Content Processing Capabilities

Topics

Architecture and Common CustomizationsCustomizing Content ProcessingCreating Search VerticalsCustomizing FederationCustomizing Search Web PartsSearch-Driven ApplicationsUsing FAST Query Language (FQL)

1. Common search – Keyword Syntax2. Extended search– FAST Query Language

Query Language Alternatives

SharePoint Server 2010

FAST Search for SharePoint – FQL SharePoint Server Search

Keyword Syntax 1

2

[property-spec:]:operator(operand [,operand]* [, parameter="value"]*)

FQL Operators

Query Operator: A keyword that specifies an operation to perform

Operator Type Keywords

Boolean AND, ANDNOT, OR, ANY, NOT, COUNT, RANK, XRANK

Proximity NEAR, ONEAR

Numeric FLOAT, INT, DATETIME

String WEIGHT, WILDCARD, MODE

Boundary Match STARTS-WITH, ENDS-WITH, EQUAL

Location Awareness Geo-search using sortformula

Formula sorting allows complete control of distance calculations (standard or ‘warped’ geometries)

Content Processing Search/Index Query

Sort results by distance from

end-user location

(50,100)

Index

External data

ABC

XYZ

123

Geo data

Geo specific processing

Geo

Pos_1 Pos_2

Lat Lon LonLat

Result

query=hotel&sortby=+[formula:sqrt(pow(50-latitude,2)+pow(100-longitude,2))]

Distance(from 50,100)

XRANK demo

Demo

Query Language ExpressivenessSoft boost using the Xrank operator

Boost all documents that contain the name

Or, in the FAST Query Language:

Single (documents containing either Mary, Diane or both gets a boost of 5000):Xrank(string(”performance”), or(person:string(”diane tibbott”), person:string(”mary baker”)), boost=5000))

Accumulative (documents with both Mary and Diane gets a boost of 10000):Xrank(Xrank(string(”performance”), person:string(”diane tibbott”), boost=5000), person:string(”mary baker”), boost=5000)

Diane Tibbott

Boost all documents that contain the nameMaryBaker

Introducing FAST Search for SharePointA new choice for enterprise search that eliminates compromise

Productivity Search Experience

Search Driven Applications

A Single, Cost-effectiveInfrastructure

Customized productivity search and search driven applications

Social, Conversational++, Visual, Contextual

Connectivity+, Security, Relevance+, Performance, Administration+, Scale+

+

SharePoint Enterprise CAL + FAST Search for SharePoint server license

SharePoint Standard CAL + SharePoint server license

General Productivity searchLight customization

Social, Conversational

Connectivity, Security, Relevance, Performance, Administration, Scale

SharePoint 2010 search and FAST search comparison

…2010 search is simple and powerful…FAST gives the most customization…you can win big with search customization

Working with Federation: http://technet.microsoft.com/en-us/library/bb905377.aspxBlog:http://blogs.msdn.com/knewton

Summary

Play the Microsoft Office & SharePoint Track Tag Contest

Download the Microsoft Tag ReaderOpen the internet browser on your mobile phone and visit http://gettag.mobi

Come to the Expo Hall – Yellow Section OSP Info Desk for Official Rules & Collect Additional Tags from all OSP Track Sessions, Speakers and Expo Hall!

Grand Prize (1)Xbox 360 Prize Package

and Microsoft® Office 2010

Daily Prizes40 copies of

Microsoft® Office 2010

47

Q&A

jeff.fried@microsoft.comkeenan.newton@microsoft.com http://blogs.msdn.com/knewton

Track ResourcesFor More Information – http://sharepoint.microsoft.com SharePoint Developer Center – http://msdn.microsoft.com/sharepointSharePoint Tech Center – http://technet.microsoft.com/sharepointOfficial SharePoint Team Blog – http://blogs.msdn.com/sharepoint

Related ContentBreakout Sessions – See Conference Guide for full list of OSP Track SessionsInteractive Sessions – OSP Track has 10 Interactive Sessions – OSP01-INT – OSP10-INTHands-on Labs – OSP01-HOL – OSP20-HOLProduct Demo Stations – Yellow Section, OSP

Office 2010, SharePoint 2010, Project Server 2010, Visio 2010 have kiosks and demos

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

Complete an evaluation on CommNet and enter to win!

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st

http://northamerica.msteched.com/registration

You can also register at the

North America 2011 kiosk located at registrationJoin us in Atlanta next year

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

JUNE 7-10, 2010 | NEW ORLEANS, LA