Cube Process Scheduling Recommendations
IT Analytics for Symantec Data Loss Prevention 3.0 extracts data from the Oracle DLP Enforce database(s) on a scheduled basis. The extracted data is then stored in multi-dimentional cubes within the Microsoft Analysis Services database, that once processed, act as the data sources for the reports and dashboards in IT Analytics.
The frequency of the cube processing schedules will determine how current the data in the cube is. Depending on business requirements, this frequency may vary, but the general recommendation for cube processing is once a day for some cubes and weekly for others (as described below). Note that there are several variables that affect the duration of cube processing tasks but the two major factors are:
- Hardware specifications of the SQL Server hosting Analysis Services
- Amount of data being processed (i.e. overall size of the Oracle DLP database)
The lower the hardware specifications of the SQL server and the greater amount of data to process, the more time it will take and vice versa. To optimize cube processing performance, it is recommended that you create two separate tasks that will process cubes on two different schedules, per the list grouping below:
Group 1 Cubes (Process Daily) | Group 2 Cubes (Process Weekly) |
---|---|
DLP Incident Summary Cube | DLP Incident Details Cube |
DLP Discover Incident Summary Cube | DLP Discover Incident Details Cube |
DLP Endpoint Incident Summary Cube | DLP Endpoint Incident Details Cube |
DLP Network Incident Summary Cube | DLP Network Incident Details Cube |
DLP Agent Status Cube | DLP Policy History Cube |
DLP Incident Status History Cube | |
DLP Discover Scans Cube | |
DLP Incident History Cube | |
DLP User Action Audit Cube | |
DLP Network Statistics Cube |
The first task will include all the DLP summary cubes and be processed daily. This should provide enough information on a daily basis to give end users the visibility they need into their DLP environment. The second process includes the more detailed and historical cubes which only need to be processed weekly. This orientation helps to expedite cube processing and ensure the right data is available for end users.
Cube Processing Benchmarks (General Estimates)
Your business requirements may stipulate that data must be updated daily, as such all cubes may need to be processed each day. Using the cube groupings outlined above, you can run these tasks sequentially on a daily basis, however be careful to allow enough time for the first task to finish before the next one begins. Again, depending on hardware resources and amount of data in the DLP database, this will take some trial and error to optimize completely. To help you start this task, the tables below provide administrators some general benchmarking estimates for cube processing (based on environment size and hardware specifications) in order to determine the approximate times necessary for your environment.
NOTE: The processing intervals listed below are estimates ONLY. Your times will vary based on the hardware specifications and amount of data in your environment. These times are offered as general guidelines only.
Incident Count | |||
Small | Medium | Large | |
Endpoint Incidents | 5,000 | 10,000 | 4,000,000 |
Network Incidents | 40,000 | 500,000 | 4,000,000 |
Discover Incidents | 10,000 | 50,000 | 1,000,000 |
Hardware Component | |||
Small | Medium | Large | |
Hardware Type | Virtual | Virtual | Physical |
Processors | Quad Core | Eight Core | 64 Core |
RAM | 8GB | 8GB | 256GB |
The table below provides guidance on the impact the SQL Server hardware (as defined above) has on the time it takes to process a given cube.
IT Analytics DLP Cubes | Processing Times per SQL Hardware Options | ||
Small | Medium | Large | |
DLP Administrative Events Cube | 10s | 10min | 30min |
DLP Scans Cube | 30s | 5min | 30min |
DLP Agent Status Cube | 20s | 20s | 1hr |
DLP Network Incident Summary Cube | 3mins | 30min | 2hrs |
DLP Discover Incident Summary Cube | 4min | 5min | 3hrs |
DLP Endpoint Incident Summary Cube | 3min | 1min | 3hrs |
DLP Incident Summary Cube | 3min | 30min | 3.5hrs |
DLP Incident Status History Cube | 30min | 2hr | 4.5hrs |
DLP Messages | 5s | 1hr | 3hrs |
DLP Network Incident Details Cube | 3min | 1hr | 5hrs |
DLP Discover Incident Details Cube | 4min | 5min | 5hrs |
DLP Endpoint Incident Details Cube | 3min | 1min | 5hrs |
DLP Incident Details Cube | 3min | 1hr | 5hrs |
DLP Incident History | 3min | 1hr | 5hrs |
DLP Policy History Cube | 1min | 45min | 4hrs |