DEV11SharePoint Search Extensibility 

Mike FitzmauriceSenior Technical Product ManagerMicrosoft Corporationmikefitz@microsoft.com

Goals

• This is about how to:–customize the out-of-box search experience–extend the search infrastructure–use the search APIs within your own applications

• Fair warning… this is not:–a search overview session–a manageability session– focused on parsing out product features by SKU

Agenda

• Quick overview of how SharePoint Search works• Customizing the built-in user experience• Extending Search• Reusing Search from remote applications

Agenda

• Quick overview of how SharePoint Search works• Customizing the built in user experience• Extending Search• Reusing Search from remote applications

Goals for SharePoint Sever Search

• Multiple repositories, including business data, people• Relevance, according to enterprise rules• Security everywhere• Customizable user experience• Performance• Part of an integrated strategy

– Windows Search for the Desktop– Microsoft Office SharePoint

Server for Search– Windows SharePoint Services– Exchange Server– SQL Server

...

...

X

User Requests

Load Balancer

Query servers

Web front ends

...Crawling

Content databases

Indexer

Propagation of indexes

...

External content

Indexer

SharePoint Search ArchitectureSearch topology model

User Requests

Load Balancer

Web front ends

Query servers

Propagationof indexes

Content Sources

SQL Server

...

...

X

User Requests

Load Balancer

Query servers

Web front ends

...Crawling

Content databases

Indexer

Propagation of indexes

...

External content

...

...

X

User Requests

Load Balancer

Query servers

Web front ends

...Crawling

Content databases

Indexer

Propagation of indexes

...

External content

...

...

X

User Requests

Load Balancer

Query servers

Web front ends

...Crawling

Content databases

Indexer

Propagation of indexes

...

External content

Index Files

Crawling

Property Store

Keep these concepts in mind…

• Content sources– OOB support for Web Content, NTFS File Shares, Exchange

Public Folders, Lotus Notes Databases, SharePoint Content, SharePoint profiles, Business Data Catalog

– More repositories can be accessed using custom protocol handlers

• Scopes– Used to create logical slices of the corpus– Scopes can span multiple content sources– Scope by Property, Site, Content Source, URL

• Properties

End User Experience

• Search Center– Clean UI, Search Site Template, Web Parts

• New Search Syntax– Implicit AND, Property Search, +/-, Advanced Search

• New Results Features– Query Correction, Duplicate Collapsing, Hit Highlighting, Best

Bets, Auto Definitions

• Improved Alerting and Search RSS Feeds• Security Trimmed Results

Agenda

• Quick overview of how SharePoint Search works• Customizing the built-in user experience• Extending Search• Reusing Search from remote applications

CoreResults

MatchingKeywordsAndBest Bets

HighConfidence

ActionLinks

SearchTabs

Search Center

Search Box

PaginationStatistics

Search Tabs

• 2 tabs are provided out of the box– All Sites tab (standard scope)– People tab

• You can add more– Custom tabs can be added for additional data types /

custom search experience– Consists of a layout page and a results page

• Not available in Search SKU

Search Web Parts

• 9 OOB web parts including – Search Box– Core Results– Action Links– Share data through hidden object

• Web part properties such as– Formatting– Stemming– # of results returned – XSLT

Customize UI With XSLT

• Appropriate for scenarios requiring– Change results layout, look and feel– Pivoting using the keyword syntax

• Web Part property• Power of XSLT

– Formatting– Logic – Math

Recap of Options

• Add/Remove/Change Tabs• Add/Remove OOB Web Parts• Change Web Part Properties

– Change XSLT

• Adding Custom Web Parts– OOB Web Parts use sealed hidden object

• You can’t add new Web Parts that depend on it, nor can you replace one of ours with one of yours

• You *can* replace all of them with a set of your own Web Parts

– You *can* use URL request parameters we pass between pages

Agenda

• Quick overview of how SharePoint Search works• Customizing the built-in user experience• Extending Search• Reusing Search from remote applications

Extending Search

• Customizing the Query and Results– Query Object Model

• Customizing the Index– Index custom data - Protocol Handlers, IFilters, BDC– Custom Query Time Security Trimmer

Query Object Model

• Use query OM– To build custom search UI web parts– To have direct access to query and results properties– To invoke custom queries

• 2 types of query syntaxes– Keyword– SQL

• URL Request Parameters

• Use Case– End user– Leverage customization

• Simple and easy to use– Avoids parsing and construction of SQL– Required and excluded term support

• Bike –fitness • SharePoint search -authur:”Richard Riley”

– Filter by any string or integer property• Consistent property:value syntax across Office, Windows and Live

search• http://msdn2.microsoft.com/en-us/library/ms497636.aspx

Query SyntaxKeyword Syntax

gallery hinges –brass site:http//supportdesk scope:Products

Use CaseAdvanced search solutionsComplex queries

Consistent SQL across enterprise and desktop

Query SyntaxSQL Syntax

SELECT URL, Title, DescriptionFROM portal_content..SCOPE()WHERE FREETEXT(‘gallery hinges’)AND SITE = “http://supportdesk”AND SCOPE = “Products”AND NOT CONTAINS(‘brass’)

Full strength, complete coverage– freetext()– Arbitrary groupings for AND, OR, NOT– CONTAINS()– LIKE– ORDER BY ASC | DESC– Removed in MOSS 2007

• UNION ALL • MATCHES• SELECT *• COALESCE TABLE  

• http://msdn2.microsoft.com/en-us/library/ms493660.aspx

Query SyntaxSQL Syntax

• Use Case– Launching a URL in custom application– Save searches

• Content– results.aspx?k=fish

• Scopes– results.aspx?k=fish&s=BBC

• Sort– results.aspx?v=date – results.aspx?v=relevance

• Page– results.aspx?start=21

Query SyntaxURL Request Parameters

Query Object ModelFeatures

• Managed code API• Single request – multiple results

Result Types• Relevant results• High confidence results• Special terms• Definitions

Optional parameters• # of Sentences in Summary• Implicit - AND/OR• Number of results• Ignore noise words• Enable stemming• Language

SharePoint Search Query Model

Query Engine

Query Engine

DefinitionsResultTable- IDataReader

Relevance

High Confidence

Best Bets

Query Object Model & Web Service

Keyword syntax SQL Syntax

Query

Client Application

Browser or ClientResults UI

• Same flow, keyword or SQL syntax– Set up query text and options– Execute()– Retrieve a set of IDataReader objects

Query Object Model

Indexing Custom Data

• Expose data as HTML/XML via HTTP• Develop a Protocol Handler to gather data from custom

repositories– Connects to a content source and enumerates the

documents– Ships with support for

• Web Content, NTFS File Shares, Exchange Public Folders, Lotus Notes Databases, SharePoint Content, SharePoint profiles, Business Data Catalog

– Interfaces are the same as in SPS 2003– MSDN: ISearchProtocol and IUrlAccessor– New registry path:

• Register PH under HKLM\Software\Microsoft\Office Server\12.0\Search\Setup\ProtocolHandlers

• http://msdn.microsoft.com/library//spssdk/html/_introduction_to_a_protocol_handler.asp

Indexing Custom Data

• Develop IFilter for custom file formats–Reads a document and extracts the plain text

content and properties for the indexer to index–Support reading from streams

Business Data SearchIndexing Structured Data Without Custom Code

• Highest integration without writing code• Information in LOB systems is often hard to access• MOSS 2007 can bring that data to your users

– Data is accessed through the Business Data Catalog– Exposed to many features in SharePoint

• Once a web service or a relational data source was registered with the BDC, search can easily index the data– No need to write code– Highly customizable results– Integrated with scopes and search center

Security

• Query-time security trimming (Same as SPS 2003)– File shares, WSS/SPS 2003, Exchange, Lotus Notes

(via mapping)

• Support for custom security trimming– ISecurityTrimmer interface

• Microsoft.Office.Server.Search.Query namespace– Implement Initialize() and CheckAccess()– Add the custom security trimmer assembly to the GAC– Register the security trimmer using stsadm command

Agenda

• Quick overview of how SharePoint Search works• Customizing the built in user experience• Extending Search• Reusing Search from remote applications

Query Web Service• Use Case

– Remote application • ASP.NET 2.0, .NET CF, or any other app…

– Office Research Pane– Windows Search

• Similar features as query object model• Results in XML and data table formats• http://o12server/_vti_bin/search.asmx• http://msdn2.microsoft.com/en-us/library/ms559279.aspx

Query Web Service• Methods

– Query – returns results as XML format• Research and Reference Service-

compatible XML• Use in the Office Research pane

– QueryEx – • Returns .NET DataSet• Full set of result types, features

– GetSearchMetaData – Retrieve available properties and scopes

– Registration– Status

Summary / Call to Action

• The user interface is easily customized to achieve a different look and feel

• It’s easy to extend search to develop custom applications, local or remote

• You can plug in just about any data source• Get out there and have at it

Additional Information

• SharePoint Team Blog– http://blogs.msdn.com/sharepoint

• SharePoint Server 2007 Developer Portal– http://msdn.microsoft.com/office/sharepoint

• Product information– http://www.microsoft.com/sharepoint