How to test and verify your ALFA policies
Unit testing adds a lot of value to the software development process by ensuring code behaves as intended, and by protecting against regressions. These same benefits can be brought to your ALFA policy code.
This guide will walk you through setting up Enforcer to run tests against your policies. It is assumed that you have a familiarity with unit testing. Examples in this guide will be written using xUnit and FluentAssertions. Complete code for this example can be found on GitHub at https://github.com/RockSolidKnowledge/Samples.Enforcer
Consider the following ALFA policy code which controls door access for AcmeCorp. According to this policy, employees may open the main door during office hours. Additionally, only IT Administrators may open the server room door at any time. Whenever either door is opened there is an obligation to capture an audit trail.
namespace AcmeCorp.DoorPolicy
{
namespace Audit
{
attribute Subject { id = "audit.subject" type = string category = subjectCat }
attribute Message { id = "audit.message" type = string category = subjectCat }
attribute When { id = "audit.when" type = dateTime category = subjectCat }
obligation Log = "Audit.Log"
}
import Oasis.Functions.*
import Oasis.Attributes.*
policyset global
{
apply firstApplicable
policy doorAccess
}
// Controls access to all doors in the building
policy doorAccess
{
target clause ResourceType == "door" && Action == "open"
apply denyUnlessPermit
// Controls access to the main door, employees can open door during office hours only
rule mainDoor
{
permit
target clause Resource == "mainDoor"
condition Subject.Role == 'employee' &&
CurrentTime >= "08:00:00":time &&
CurrentTime < "18:00:00":time
on permit
{
obligation Audit.Log
{
Audit.Subject = Single(Subject.Identifier)
Audit.Message = "Opened main door"
Audit.When = Single(CurrentDateTime)
}
}
}
rule serverRoom
{
permit
target clause Resource == "serverRoomDoor"
condition Subject.Role == 'ITAdmin'
on permit
{
obligation Audit.Log
{
Audit.Subject = Single(Subject.Identifier)
Audit.Message = Single(Resource) + " was accessed"
Audit.When = Single(CurrentDateTime)
}
}
}
}
}
Test fixture setup
We will begin by setting up Enforcer. Create a ServiceCollection, add Enforcer and configure the policy enforcement point. Build the service provider, and then use it to request our policy enforcement point for testing. In this example, the policy code will be an embedded resource in the assembly.
We are also providing an OutcomeActionHandlerFactory and an in-memory logger, these will be explained in detail later in this document.
public class DoorAccessPolicyPolicyTests
{
private readonly ServiceProvider serviceProvider;
private readonly AuditObligationHandler obligationHandler;
private const string Licensee = "DEMO";
private const string LicenseKey = "Obtain a demo license key at https://www.identityserver.com/products/enforcer";
private const string RootPolicyName = "AcmeCorp.DoorPolicy.global";
private const string EmbeddedPolicyRoot = "PolicyTestingSample.Policies";
public DoorAccessPolicyPolicyTests()
{
var obligationHandlerFactory = new TestObligationHandlerFactory();
obligationHandler = (AuditObligationHandler) obligationHandlerFactory.Create();
var serviceCollection = new ServiceCollection();
serviceCollection
.AddEnforcer(
RootPolicyName, o =>
{
o.Licensee = Licensee;
o.LicenseKey = LicenseKey;
o.PolicyInformationPointFailureBehavior = PolicyInformationPointFailureBehavior.Continue;
o.OmitStandardPIPs = true;
})
.AddPolicyEnforcementPoint(o => o.Bias = PepBias.Deny)
.AddEmbeddedPolicyStore(this.GetType().Assembly, EmbeddedPolicyRoot)
.AddOutcomeActionHandlerFactory(obligationHandlerFactory);
serviceProvider = serviceCollection.BuildServiceProvider();
}
private IPolicyEnforcementPoint CreateSystemUnderTest()
{
return serviceProvider.GetService<IPolicyEnforcementPoint>();
}
}
Adding some tests
The first thing we need to verify about our policy is that we can find it and it compiles correctly. To do this we create the policy enforcement point and evaluate it. We do not need to be concerned with providing attributes at this point, exceptions will be thrown if the policy cannot be found or fails to compile. For convenience, we can capture and print out compilation errors with the test failure message.
[Fact]
public async Task DoorAccessPolicy_ShouldCompileWithoutErrors()
{
try
{
var sut = CreateSystemUnderTest();
var outcome = await sut.Evaluate();
}
catch (PolicyCompilationException e)
{
var errors = new StringBuilder();
foreach (PolicyCompilationIssue issue in e.Errors)
{
errors.AppendLine($"Compiler Error at ({issue.Line}:{issue.Column}): {issue.Message}");
}
Assert.True(false, errors.ToString());
}
}
Testing the policies
In order to execute the policies in this example we need some additional moving parts. First, we need to provide values for the attributes required by the policies. The policies also have obligations which must be performed, so we need to provide an OutcomeActionHandler. The OutcomeActionHandlerFactory registered in the fixture setup will provide this. Failing to perform obligations will always result in a Deny result.
Providing attribute values
When we evaluate the policy enforcement point, the call accepts an IAttributeValueProvider. It is through this interface that attribute values are resolved. While you could provide your own implementation, Enforcer has a DynamicAttributeValueProvider which allows attributes to be configured with literal values, ideal for the context of a unit test. Standard Enforcer attributes have a .Net definition and can be directly used, see Oasis Attributes for more details.
The following test uses a DynamicAttributeValueProvider to execute the policy. The test verifies that we have a deny result when an employee attempts to open the main doors outside of office hours, as expected.
[Fact]
public async Task Employee_OpeningMainDoorOutsideOfOfficeHours_ShouldBeDenied()
{
string subjectId = "alice";
string role = "employee";
string resourceType = "door";
string resourceAction = "open";
string resourceName = "mainDoor";
Time timeOfDay = new Time(20, 00, 00);
DateTime timeNow = DateTime.Now;
var request = new DynamicAttributeValueProvider();
request
.AddString(Rsk.Enforcer.Oasis.Attributes.Subject.Role, role)
.AddString(Rsk.Enforcer.Oasis.Attributes.Subject.Identifier, subjectId)
.AddString(Rsk.Enforcer.Oasis.Attributes.ResourceType, resourceType)
.AddString(Rsk.Enforcer.Oasis.Attributes.Action, resourceAction)
.AddString(Rsk.Enforcer.Oasis.Attributes.Resource, resourceName)
.AddTime(Rsk.Enforcer.Oasis.Attributes.CurrentTime, timeOfDay)
.AddDateTime(Rsk.Enforcer.Oasis.Attributes.CurrentDateTime, timeNow);
var sut = CreateSystemUnderTest();
var outcome = await sut.Evaluate(request);
outcome.Outcome.Should().Be(PolicyOutcome.Deny);
}
Time-based tests
An important point to note in the previous example is that the policy result depends on the time of day. In a conventional unit test, we would avoid coupling the test to the actual time of day and instead mock a dependency to provide a value for the time that we can control. Enforcer is no different in this regard, by default Enforcer has an attribute value provider for environmental values such as the current date and time. In order for the policy test to not be coupled to the actual time of day, we need to instruct Enforcer to exclude these default attribute values so that we can provide our own. This is done by setting the OmitStandardPIPs property on the EnforcerOptions during start-up.
serviceCollection
.AddEnforcer(
RootPolicyName, o =>
{
o.OmitStandardPIPs = true;
// And other options
})
Handling and testing obligations and advice
If we take the previous test and modify it to be within office hours, with the current setup this will still result in a deny result because the policy has an on permit obligation. We need to create a handler for the obligation in order to continue.
The following snippet shows an OutcomeActionHandler for use in these tests, for more details see Implementing a Custom Outcome Action Handler
internal class AuditObligationHandler : OutcomeActionHandler<AuditLogArguments>
{
public List<AuditLogArguments> Invocations { get; } = new List<AuditLogArguments>();
public override string Name => "Audit.Log";
public AuditObligationHandler()
{
Invocations?.Clear();
}
protected override Task Execute(AuditLogArguments parameters, IEnforcerLogger evaluationLogger)
{
Invocations.Add(parameters);
return Task.CompletedTask;
}
}
public class AuditLogArguments
{
[PolicyAttribute(PolicyAttributeCategories.Subject, "audit.subject")]
public string Subject { get; set; }
[PolicyAttribute(PolicyAttributeCategories.Subject, "audit.message")]
public string Message { get; set; }
[PolicyAttribute(PolicyAttributeCategories.Subject, "audit.when")]
public DateTime? When { get; set; }
}
Because Enforcer is in control of creating OutcomeActionHandler instances during start-up, we need to be able to obtain the same instance in tests to be able to verify the handler is invoked with the correct arguments.
To do this, we will create an OutcomeActionHandlerFactory which returns the same instance each time. Enforcer allows us to register a specific factory instance during start-up.
public class TestObligationHandlerFactory : OutcomeActionHandlerFactory
{
private OutcomeActionHandler instance = new AuditObligationHandler();
public override OutcomeActionHandler Create()
{
return instance;
}
public override string Name => "Audit.Log";
}
The following example illustrates testing a policy with obligations.
[Fact]
public async Task Employee_OpeningMainDoor_ShouldPermitAndCaptureAuditTrail()
{
string subjectId = "alice";
string role = "employee";
string resourceType = "door";
string resourceAction = "open";
string resourceName = "mainDoor";
Time timeOfDay = new Time(15, 00, 00);
DateTime timeNow = DateTime.Now;
var request = new DynamicAttributeValueProvider();
request
.AddString(Rsk.Enforcer.Oasis.Attributes.Subject.Role, role)
.AddString(Rsk.Enforcer.Oasis.Attributes.Subject.Identifier, subjectId)
.AddString(Rsk.Enforcer.Oasis.Attributes.ResourceType, resourceType)
.AddString(Rsk.Enforcer.Oasis.Attributes.Action, resourceAction)
.AddString(Rsk.Enforcer.Oasis.Attributes.Resource, resourceName)
.AddTime(Rsk.Enforcer.Oasis.Attributes.CurrentTime, timeOfDay)
.AddDateTime(Rsk.Enforcer.Oasis.Attributes.CurrentDateTime, timeNow);
var sut = CreateSystemUnderTest();
var outcome = await sut.Evaluate(request);
outcome.Outcome.Should().Be(PolicyOutcome.Permit);
var auditLogs = obligationHandler.Invocations.ToArray();
auditLogs.Length.Should().Be(1);
auditLogs[0].Subject.Should().Be(subjectId);
auditLogs[0].When.Should().Be(timeNow);
}
Troubleshooting policy execution
Sometimes tests policy fail unexpectedly. When this happens, it can be useful to see a trace of how Enforcer evaluates a policy to determine the cause of the unexpected result.
Enforcer provides a logger for this purpose. Create an instance of TestingLogger and register it as the logger factory during start-up. The TestingLogger holds all log messages in the LogMessages property, making it easy to inspect under the debugger or print the messages to your test output window.
The sample below shows how to wire-up the TestingLogger.
public class DoorAccessPolicyPolicyTests
{
private readonly ServiceProvider serviceProvider;
private readonly TestingLogger logger;
private const string RootPolicyName = "AcmeCorp.DoorPolicy.global";
private const string EmbeddedPolicyRoot = "PolicyTestingSample.Policies";
public DoorAccessPolicyPolicyTests()
{
var serviceCollection = new ServiceCollection();
logger = new TestingLogger();
serviceCollection
.AddLogging()
.AddEnforcer(
RootPolicyName, o =>
{
// Options
})
.AddLoggerFactory(logger)
// Additional configuration
;
serviceProvider = serviceCollection.BuildServiceProvider();
}
}