Introduction

The JS7 - Performance Tests for JOC Cockpit article explains the metrics and purpose of JOC Cockpit performance tests.

Test cases are available for download from the JS7 - Self-Test - Getting Started article.

Test cases implement a few catch hooks to prevent users from overloading their machines.

• By default the overall number of parallel sessions is limited to 16384 for a single JOC Cockpit instance. The limit is configurable.

• By default user confirmation is required for each next iteration. The confirmation is configurable.

Inventory Objects

After import of JS7 - Self-Test test cases, users find the following objects in their JOC Cockpit inventory that are related to performance tests.

• Self-Test
  • Controller
    • Job Resources
      • Self-Test-AuthenticationSettings
  • Performance-Tests
    • Controller
      • Workflows
        • Self-Test-Performance-JOC-Cockpit
        • Self-Test-Performance-Starter-JOC-Cockpit
      • Resource Locks
        • Self-Test-Performance
• Self-Test-User
  • Automation
    • Calendars
      • Self-Test-AllDays
  • Performance-Tests
    
    • Automation
      
      • Schedules
        • Self-Test-Performance-JOC-Cockpit-High
        • Self-Test-Performance-JOC-Cockpit-Low
        • Self-Test-Performance-JOC-Cockpit-Medium

Workflows

Test Run Starter Workflow: Self-Test-Performance-Starter-JOC-Cockpit

The workflow implements the starter for test runs, this is the workflow to which users add a single order to run a test.

• The Test Run Starter Workflow will not execute the test case itself but will add orders to one of the Test Run Workflows that implement the test run.
• No changes to the workflow are required with the one exception that users should assign the jobs in the workflow a Standalone Agent or Subagent Cluster.
• The Test Run Starter Workflow can be executed with any Agent on Unix or Windows platforms.
  
  

Explanations:

• Agent assignment: by default all jobs in the Test Run Starter Workflow are assigned a Standalone Agent with the name: self-test-agent.
  • Users can use the "Search and Replace" button in the right upper corner of the Workflow Editor. This allows to replace Agent assignments to jobs in the workflow.
  • Users can add the self-test-agent alias name for an existing Standalone Agent in the Manage Controllers/Agents page.
• Job Resources
  • The workflow is assigned the Self-Test-AthenticationSettings job resource, for adjustments see JS7 - Self-Test - Getting Started.

Workflow Variables:

Test Run Workflow: Self-Test-Performance-JOC-Cockpit

The workflow is the implementations of a Test Run Workflow that is intended for use with Standalone Agents.

• Users can use the "Search and Replace" button in the right upper corner of the Workflow Editor. This allows to replace Agent assignments to jobs in the workflow.
• Users can add the self-test-agent alias name for an existing Standalone Agent in the Manage Controllers/Agents page.
• Variables in the workflow need no adjustment as they will be passed by the Test Run Starter Workflow.

Explanations:

• Agent assignment: by default all jobs in the workflow are assigned the Standalone Agent with the name: self-test-agent.
  • Users can assign any Standalone Agent or Agent Cluster that should run the test case.
  • Users can add the self-test-agent alias name for an existing Standalone Agent in the Manage Controllers/Agents page.

Resource Locks

Resource Lock: Self-Test-Performance

The resource lock is used to prevent parallel execution of test runs.

Calendars

Calendar: Self-Test-AllDays

The JS7 - Self-Test includes a global calendar that is used by schedules related to performance tests:

• Users who run performance tests manually should consider that the calendar must exist, but is not used. The schedules provided are not configured for automated execution with the Daily Plan.
• Users who wish to run performance tests on a regular basis, can copy and adjust the calendar to specify applicable days.

Schedules

A number of schedules are provided that hold reasonable parameterization per test scenario (low, medium, high).

Schedules are not configured for consideration by the JS7 - Daily Plan Service and by default will not make it for automated execution with the Daily Plan.

Users who wish to automate execution of performance tests can copy the schedules and can configure use with the Daily Plan, see JS7 - Schedules.

Available schedules reference the Self-Test-Performance-Starter-JOC-Cockpit Test Run Starter Workflow and hold the parameterization for test scenarios.

Schedule: Self-Test-Performance-JOC-Cockpit-Low

The schedule holds the parameterization for low performamce requirements which includes a maximum of 128 parallel  user sessions.

Schedule: Self-Test-Performance-JOC-Cockpit-Medium

The schedule holds the parameterization for medium performamce requirements which includes a maximum of 512 parallel user sessions.

Schedule: Self-Test-Performance-JOC-Cockpit-High

The schedule holds the parameterization for high performamce requirements which includes a maximum of 2048 parallel user sessions.

Starting a Test Run

To start a test run, users add an order to the Self-Test-Performance-Starter-JOC-Cockpit Test Run Starter Workflow like this:

This will bring forward a popup window for order parameterization like this:

• In the order parameterization popup window users can click the Assign parameterization from Schedule link.
• This will open a dropdown list from which users can choose a schedule. Parameterization will be copied from the selected schedule.

Having selected a schedule, related variables are displayed with the popup window for parameterization:

• It is preferable to use one of the schedules provided with JS7 - Self-Test to find a matching parameterization for the intended test scenario (low, medium, high).
• Users can modify variables at their will.
  
  

In most situations parallel execution of test runs should be avoided. To this purpose the Self-Test-Performance resource lock is used that will make parallel orders for the Test Run Starter Workflow wait until the lock becomes free. Users who which to run performance tests in parallel, can increase the resource lock's capacity to the number of parallel test runs like this:

Stopping the Test Run

You were wrongly assuming that iteration 14 from a base of 2 means 28 orders and you are surprised to find 16384 orders being added to your test run?

A few things users should not do in this situation:

• Do not shutdown the JS7 products. Not the Agent, not the Controller, not JOC Cockpit.

Users can apply the following actions:

• Navigate to JOC Cockpit's Workflows view and to the Self-Test-Performance-Starter-JOC-Cockpit Test Run Starter Workflow. Suspend the workflow from its action menu.
• Similarly suspend the Test Run Workflow from its action menu.
  
  

Should JOC Cockpit become slow due to high order load, then it might be hard to use the action menu specifically. In this situation, use the Suspend All butoon on top of the page which acts as an emergency stop. The operation will freeze any orders and will prevent them from executing jobs.

When orders are frozen, they can be cancelled from the Orders Overview view that is available by clicking the number related to the order state in the Dashboard view. When cancellation of frozen orders is performed then orders will not leave the Test Run Workflow immediately. Instead the workflow has to be resumed from a similar Resume All operation. This will make orders wake up and leave the Test Run Workflow.

Resources

• JS7 - Self-Test - Getting Started
• JS7 - Performance Tests
• JS7 - Performance Tests for JOC Cockpit