Foldr Search – Setup

Posted on 18th August 2017

This feature is currently classified as BETA

Introduction

Foldr appliance update v4.3 introduces a powerful search function that allows users to perform searches across multiple storage locations at once.  Users can search for files by name, filter results by modified date or search for specific key words within files.  Foldr Search contains an optional Optical Character Recognition (OCR) engine to read text from images, such as scanned documents or PDFs.

Search results are returned by querying an index held within the appliance, which provides incredibly fast search results, regardless of the number of storage areas being indexed and searched.  The index itself is built up from crawl jobs that are run on a scheduled basis.  The initial crawl process of a share may take some time depending on the amount of data to be indexed, however subsequent crawls will only index changed or new files so the index jobs will complete much faster.

By default, the search feature is disabled and must be enabled and configured by the administrator before it is available for use.

Currently, only SMB shares can be indexed and searched by users, however other platforms such as Google Drive and OneDrive will be included in the search results as part of a future appliance update.

Foldr automatically ensures only appropriate search results are returned to users, based upon the shares that are available to them under My Files and by analysing the file server backend NTFS permissions / ACLs.

System Requirements & Deploying a Dedicated Search Appliance

The Search role can be resource intensive both in terms of CPU and memory and as such it is strongly recommended that a separate virtual appliance is deployed specifically to host the search indexes and perform crawl operations.  If regular client access and search is hosted upon a single appliance, it may have an impact on the user experience when performing regular file access operations, even when increasing the specifications of the VM.  The following minimum specifications are recommended for the Foldr appliance that is going to be hosting the search role:

2 vCPU
4GB RAM

If you are can provide more CPU / RAM resources to the search appliance beyond the specification above, the crawl process will, within reason, consume most of compute resource it is provided with during the index operation. The above specification is the recommended minimum for Search to operate correctly.

This article will describe configuring two Foldr appliances.  One will act as our primary client access / infrastructure appliance, and the other our Search appliance.  In an existing installation, the primary appliance is the virtual machine that is currently being accessed by users.

Enabling and Configuring the Index Service

This should be done on the Search Appliance

Within Foldr Settings, navigate to the Search tab and enable the Index Service.

Setting the Crawl Schedule

This should be done on the Search Appliance

Enable scheduled indexing and choose a suitable time to run the crawl jobs. Jobs are run sequentially, one URI / share at a time.

Specify other Foldr appliances (Trusted Servers)

This should be done on the Search Appliance

All other Foldr appliance(s) that will be using search must be entered within the Trusted Servers field with each entered onto a separate line.

This will change the configuration of the built-in firewall to allow connections from these IP addresses.

Creating a Search Core

This should be done on the Search Appliance

A ‘Core’ can be thought of as a container that holds both the configuration and the index files for one or more share paths (URIs).

Whilst the search function can host multiple cores with numerous share URIs in each, it is recommended to configure a single core with all share URIs within it.

On a multi-tenant installation, you should use one core per tenant.

1. To create a core, click the Cores tab >> + Add Core

2. Give the Core a suitable name and click ADD CORE. Please note only lowercase characters are permitted.

3. A new, empty Search Core will be created. Click + Add URI to add a share path to be indexed.

4. The Add URI dialog will be displayed. Here you must configure the address (the network path) for the shares that you wish to index, provide a suitable service account that has permission to read the files and finally set the crawl schedule. If no service account is available for selection, one can be configured within General tab >> Service Accounts. Note the username should be supplied in UPN format.

Optional – The search appliance may be pointed at an Infrastructure appliance to retrieve the service accounts along with other configuration held upon it.  This would remove the need to manually configure service accounts on the Search appliance.  See the following KB for more information

Note – Only SMB shares can be indexed. Support for search results from cloud storage providers such as Google Drive, OneDrive etc will come in a later update.

IMPORTANT – The ‘Address’ (smb share path) must match EXACTLY as the share is configured in Foldr Settings >> Shares. If the share path is configured fully qualified or using a short unqualified path it must match exactly. If shares are configured under the Shares tab using an IP address rather than DNS hostname, ensure the URIs for search are configured the same way.

5.  Build up a list of URIs to index (which are to be searchable by end users), generally this would include all SMB shares configured for user access on the client access appliance.

Adding URIs for Active Directory Home Folders

This should be done on the Search Appliance

If the %homefolder% variable is being used to present users home folders, the share path should be confirmed within in Active Directory Users & Computers > Properties > Profile tab > Home Folder > Connect and the root / top level share added as a URI within the Search Core.

If you use multiple paths for different groups of users, each should be added as a separate URI in the Core.

Example:

For the example above, you would need to add the following URI to the Search Core to allow sales users to search their home folders

Enabling Client Access (Allow users to use Search)

This should be done on the Client Access appliance

On the Foldr appliance(s) that provide client access (typically those that users directly interact with via the web, desktop and mobile apps) you must enable Client Access in Search before they can use it.

The Search tab will hidden in the web app until Client Access is enabled within Foldr Settings >> Search >> Client Access.

Enabled ‘Provide search to clients’ and specify the IP address of the appliance that is running the Search role.

Finally, provide the Core Name as configured at step 2 above and click SAVE.

Once client access is enabled, the Search tab will become available in the web app.


Note that the Client setting is replicated across a cluster.

Search Results & Permissions

As part it’s scheduled crawl operations, Foldr Search respects the backend file server permissions / ACLs. This ensures only appropriate search results are returned to a user for each search query by using the ACLs and checking which shares a user actually has access to in Foldr itself. (My Files)

For example, if a sales department specific network share URI is being crawled, only users who can see the sales share in the Foldr interface will be shown search results relating to files contained upon it.

Indexing New Files

Foldr will automatically index files as they are uploaded by users and will also update the index with regard to move and file deletion events. If new files are placed onto an SMB share outside of Foldr (i.e. from a domain bound workstation in Explorer) the files will be added to the Foldr search index when the next scheduled crawl job takes place.  You can manually start an index / crawl job at any time from the Cores tab.

Manually Starting a Crawl / Index Job

This should be done on the Search Appliance

To start a crawl job manually, click the inline button shown below from the Cores tab for the relevant URI / share path.

Note – You do not need to enable ‘force re-indexing of all files’ in order to pick up new files that may have been uploaded from outside of Foldr.

Need more help?

Get in touch with our friendly help desk who will be happy to assist you, support@foldr.io