Tag Archives: Intelligence Analysis

Graph or Social Network Analysis – what’s the difference?

Posted on by

Paul Curwell, CPP, CFE, CISSP

Common terminology sows the seeds of confusion

If you’re someone who has been involved in fraud protection, Anti-Money Laundering, Counter-Proliferation, Sanctions Evasion, anticounterfeiting (the list goes on) – basically any sort of investigation of networks, you will likely have come across concepts such as graph, link analysis, and network analysis. However, when you start to write use cases for your organisation and develop your functional requirements for technology, this starts to get messy. For those new to this area, the figure below provides an illustration of what social network analysis is:

Illustration of a social network in analyst notebook
Social Network Analysis illustration, US Dept. of Justice (2016)

Unfortunately, the terminology we use every day is the source of much confusion amongst business users (investigators, intelligence analysts, security & fraud professionals), data scientists and technologists alike, making it hard to understand the actual problem which needs to be solved by technology. To understand this space, there are three main concepts to get your head around:

  • Network Analytics: Is a term that has its origins in computer science and ICT, and is used to help model, monitor and assess the health and performance of computer networks
  • Graph Analytics: Also known as ‘Graph Technology’, this term actually refers to a type of database – the Graph Database – which stores data in the form of a ‘graph’ or network. Graph is heavily used today in the newly emerged field of Data Science.
  • Social Network Analysis: Also known as ‘link analysis’, ‘network analysis’, and a variety of other names, this methodology has been around since the 1970’s and stems from the social sciences. It uses algorithms and other methods to model and depict the behaviours of groups of entities (e.g. people, objects), attributes (e.g. the characteristics of objects, such as a person’s name), and the relationships (connections) between them. This is important as Entities typically exist as ‘networks’ in society.

The three concepts outlined above, each a distinct academic discipline, can be applied to three simple User Personas, as outlined below:

UserUse Case
IT DepartmentsUse network analytics to assess and manage the health of your IT and OT (operational technology – such as SCADA systems) networks
Data Scientists, Data EngineersUse Graph Databases to facilitate complex modelling, analysis, and other data management related tasks
Intelligence Analytsts, Investigators, Risk & Compliance OfficersPerform social network analysis to understand threat networks, such as criminal networks, organised fraud syndicates, or illicit corporate structures to assist in their identification, targeting and disruption
Three illustrative user personas for graph and social network analysis

Despite often using terminology interachangeably, we are actually referring to three distinct concepts which cause confusion when co-mingled.

What is a graph exactly?

A basic graph – whether we are talking about the way data is visualised within a graph database or as part of social network analysis – is depicted by nodes (entities) and edges (links or relationships). Fraud teams use enhanced depictions of ‘graphs’ to enrich a data with more information. Graphs (social networks) can be queried to return matching results, such as showing all individuals who are connected to a specific address in some way (e.g. home, work, family connections).

For data scientists, one attractiveness of a graph database is that large networks can be more efficiently searched or analysed compared to a Relational Database (RDBM) such as SQL Server or Teradata. There are numerous use cases for graph databases, including:

  • Entity Resolution – to determine whether two entities are actually the same based on various attributes
  • Knowledge Graphs – to help answer questions or find the answer to something
  • Product Recommendation Engines – for customers of eCommerce stores to suggest other products purchased by similar customers
  • Master Data Management
  • ICT network infrastructure monitoring
  • Fraud detection

Examples of graph databases on the market today include those produced by Neo4j, TigerGraph, AWS Neptune, Microsoft Cosmos, and many others.

Why is Social Network Analysis important for countering threat networks?

The term “Threat Network” is used by the U.S. Government when discussing any type of hostile actor (even lone actors are typically part of some social network). Examples include organised crime, nation states, organised fraud syndicates, counterfeiting syndicates, and industrial espionage networks. Without going into too much detail here, every threat network has a number of common roles which are required to achieve its objective.

Let’s say a consumer fraud ring is running a boilerroom scam to defraud elderly investors. The network needs people to manage its finances, communications, recruitment, targeting to spot vulnerable investors, scammers to actually defraud them, and managers and leaders to coordinate the scheme. This concept is illustrated below in relation to drug production and trafficking:

Organisational structure showing roles within a typical organised crime network
Illustration of various roles within a threat network (JP 3-25)

Social Network Analysis allows for visualisation of relationships and structures of all parties involved in the network, providing the ability to overlay additional information such as functions in the network. Social Science algorithms, such as Betweenness and Centrality, can be applied to social network data to identify key players or connections. These threat network vulnerabilities can then be targeted, such as through arrests or new internal controls, to disrupt threat actor activites. This concept is illustrated below:

Illustration of how a network can be disbanded (disrupted) with effective targeting
Illustration of how disrupting a network can render it ineffective (JP 3-25)

How can I perform Social Network Analysis?

Interestingly, you do not need a ‘graph database’ to perform Social Network Analysis. What you do need though is a suitable user interface for business users (e.g. investigators) which allows them to query, analyse, and interact with their data to achieve an outcome – such as identifying key players in a fraud ring. Without a suitable interface, business users will be unable to exploit the data effectively rendering it useless.

Fraud and law enforcement teams have used Social Network Analysis for decades. You can do simple Social Network Analysis on paper or a whiteboard without the use of software – this is where the term ‘link analysis’ originated from. Whilst pinboards are useful for Hollywood movies and simple networks, analysts today are swamped in data making software essential.

man in gray long sleeve suit holding a pen - social network analysis with paper and a pinboard
Photo by cottonbro studio on Pexels.com

In the late 1990’s or early 2000’s, the popular software known as Analyst Notebook was developed and is still in use today. These days, there is a proliferation of thick client and browser based software which performs this function, including Maltego, Linkurious, Palantir, Quantexa, and RipJar.

As outlined here, there is a distinct difference between the concepts of network analysis, graph and social network analysis. Each has its own use cases, methodologies, user groups and supporting software. Understanding this landscape, and how all the pieces fit together, is essential to building any sort of threat intelligence or detection analytics capability.

Further Reading

DISCLAIMER: All information presented on ForewarnedBlog is intended for general information purposes only. The content of ForewarnedBlog should not be considered legal or any other form of advice or opinion on any specific facts or circumstances. Readers should consult their own advisers experts or lawyers on any specific questions they may have. Any reliance placed upon ForewarnedBlog is strictly at the reader’s own risk. The views expressed by the authors are entirely their own and do not represent the views of, nor are they endorsed by, their respective employers. Refer here for full disclaimer.

Comparative Case Analysis: A powerful tool for typology development

Posted on by

Paul Curwell, CPP, CFE, CISSP

What is Comparative Case Analysis?

Comparative Case Analysis (‘CCA’), also known as ‘Similar Fact Analysis’, is a technique used in criminal intelligence analysis to identify similarities and support decision making (Sacha et al, 2017).

Cases can be linked in CCA through any of the following:

a) Modus Operandi (or tactics, techniques, procedures)
b) Signatures and patterns
c) Forensic evidence
d) Intelligence

College of Policing (2023), United Kingdom

CCA is useful when analysing process-based crime types where perpetrators need to follow a defined set of steps to effect the crime. Examples of such crime types include fraud and financial crime, cybercrime, money laundering and Intellectual Property Crime (e.g. counterfeiting networks).

I use CCA when developing typologies, which I then convert to analytics-based detection models which are run as part of a continuous monitoring or detection program over a dataset to detect suspect transactions, individuals/ legal entities, or behaviour.

a person pointing on to the photographs
Photo by RODNAE Productions on Pexels.com

Where can you collect cases to perform CCA?

So, you’ve worked out that CCA is appropriate to use in your situation. The next challenge is where to get your case study data from. Common sources include:

  • Indictments and statements of claim – depending on jurisdiction, these may be published by prosecutorial agencies such as the U.S. Department of Justice, or by the courts (for tips, see my article on searching Australian court records).
  • Media reports – media monitoring and other Open Source Intelligence (OSINT) capabilities are essential for any financial crime or corporate security function. For information on how to build one, look at my 101 post.
  • Industry information sharing sessions – industry groups such as the Pharmaceutical Security Institute and the Australian Financial Crimes Exchange exist for this purpose.
  • Prisoner interviews – may be performed by law enforcement, regulators, journalists or academics for publication.
  • Academic case studies, published papers and conferences
  • Examination of your own case files based on historical incidents or near-misses.

Unfortunately, it is all too common to find cases that are incomplete. If you don’t control your data (such as cases sourced from the media) your ability to improve data quality is limited – you may need to exclude incomplete cases from the CCA.

Does this article resonate with you? Please vote below or subscribe to get updates on my future articles

If you are using your own case files, consider changing your internal processes, templates and SOPs to collect the data you need in the future. If you encounter resistance, obtain buy-in from stakeholders by helping them understand what you need and why you need it.

How do you undertake Comparative Case Analysis?

CCA is an invaluable but involved process which will take time to complete. CCA has its roots in academia, particularly the social sciences (see Goodrick et al 2014), so some literature on the topic is irrelevant or too academic to be useful for typology development or intelligence analysis.

photo of women laughing
Photo by RF._.studio on Pexels.com

CCA can be undertaken individually or within a group, although doing the work individually may lead to intelligence blindspots. My high level methodology is as follows:

StepTaskConsiderations
1Define your scope, case criteria, and other considerations a) What are you attempting to achieve by performing this CCA? Is CCA the most appropriate method?
b) What risk are you seeking to mitigate and what type of case / crime type etc meets these criteria?
c) What timeframe, jurisdiction, industry / product / channel / customer type are in scope?
d) How might analytical bias arise in your methodology? How will you manage this?
2Collect your case information and prepare the data for analysisa) Refer to the ‘where can you collect cases to perform CCA?’ for suggestions
3Review each case for data quality and completenessa) Do you have sufficient information for each case?
b) Do your cases fit the criteria you defined in step 1?
c) Do you need to change your methodology?
d) Is the methodology viable with the avilable information?
e) What cases (if any) do you need to remove due to incomplete data?
4Develop a structured form or methodology to undertake the comparisona) How are you going to compare each case? I build a form or template as part of my approach which I populate with information from each case and use this for case comparison
b) What data elements do you want to compare? Details captured usually include entities (people, businesses, things such as vehicles or residences), locations and dates / times, activities (e.g. events, transactions), and attributes such as language in addition to Modus Operandi.
c) Comparison of this data enables the identification of patterns or attributes which can be used to link seemingly separate incidents together (remember criminals share with each other, a liked case doesn’t have to reflect the same individual).
5Determine where you will store your resultsa) Where will you store your captured data and analysis?
b) If dealing with large volumes of data, you may want to build a database or design a workbook in Microsoft Excel to collect the data for subsequent analysis.
6Read each case and identify each data elementa) Physically read the material for each case
b) Identify the data elements which you want to capture (step 4). One way to do this is using coloured pens or highlighters, with each colour representing a specific data element (e.g. entities).
c) Once identified, this information can be used to document your results (step 7)
7Document your resultsa) I tend to find Microsoft Word, PowerPoint or Excel is fine for this purpose, but ensure you store your CCA reports in a central location so they can be peridocially reviewed and updated.
b) An alternative is ‘visual CCA’, effectively using a visualisation tool such as Tableau or Microsoft PowerBI to analyse and present your findings (see Sacha et al 2017)
c) Ensure any assumptions, data gaps or hypotheses are clearly identified (ideally CCA is factual, so if there are information gaps you are better off leaving this blank than filling a gap with a hypothesis. The fact you have done this can get overlooked in future typology and detection model work and lead to erroneous results).
8Have an ‘independent party’ peer review or critique your worka) Having another party (e.g. team or peers, independent experts etc) not involved in original activity perform a review and challenge role.
b) This provides an opportunity to identify gaps, assumptions or conclusions in your analysis.
9Evaluate your results a) Are they complete?
b) How reliable do you think they are?
c) Are they sufficiently detailed and rigorous enough to use as a basis for typology development?
d) What if any rework do you need to do before finalising your CCA? Perform updates to your work as appropriate.
10Periodically refresh completed CCAsa) Threats such as fraud, financial crime and cybercrime are constantly changing in response to new processes, products, channels, internal controls and actions taken by fraud and security teams to mitigate these threats.
b) Implement a process to periodically reivew and update historical CCA, such as annually, and incorporate this into any detailed typologies.
Paul Curwell (2023). Comparative Case Analysis methodology, http://www.forewarnedblog.com

A simplified example of a CCA data capture template (step 4) which has been populated with fictional case information (steps 6 and 7) is shown below:

A simplified example of a CCA data capture template (step 4) which has been populated with fictional case information (steps 6 and 7).

Typology development: the next step in operationalising detection

Whilst CCA is not a pre-requisite to developing a typology, it certainly helps. When designing your CCA approach, I recommend you consider the types of data you will need to build your typology and incorporate these into your methodology (see my previous article, ‘typologies demystified‘).

Analysing Modus Operandi or TTPs requires the application of a number of intelligence analysis methods and is too big to cover here. I will write about this separately in a future post.

Further Reading

DISCLAIMER: All information presented on ForewarnedBlog is intended for general information purposes only. The content of ForewarnedBlog should not be considered legal or any other form of advice or opinion on any specific facts or circumstances. Readers should consult their own advisers experts or lawyers on any specific questions they may have. Any reliance placed upon ForewarnedBlog is strictly at the reader’s own risk. The views expressed by the authors are entirely their own and do not represent the views of, nor are they endorsed by, their respective employers. Refer here for full disclaimer.

Searching court records in Australia

Posted on by

Paul Curwell, CPP, CFE, CISSP

A subject’s legal history says a lot about their integrity and suitability

Performing any sort of counterparty due diligence requires an understanding of the “whole person” (this applies to both individuals and legal entities). In financial sector or service delivery organisations, this is referred to as a “single view of customer” and is used to manage fraud risk, credit risk and regulatory compliance.

A subject’s legal history is an important element of the ‘whole person’; without it, managers may make decisions based on incomplete or inaccurate information only to regret it later. Performing legal checks requires an understanding of Australia’s courts to develop an informed search strategy.

grey concrete court-like building
Photo by Brett Sayles on Pexels.com

Australia’s court structure

In Australia, legal matters can be brought under State / Territory or Commonwealth law, as well as other mechanisms (such as professional standards schemes which are expected to regulate their members). Some dispute mechanisms are industry based.

State or Territory courts:

  • Local Court, County Court, Magistrates Court – hears most criminal and summary prosecutions and minor civil matters (e.g. <100,000). 95% of criminal cases commence at this level.
  • District Court (excluding TAS, NT and the ACT) – hears appeals from Local Court, serious criminal cases (excluding murder, treason), civil matters typically <$750,000.
  • Supreme Court – hears serious civil cases >$750,000 and serious criminal cases (including murder, treason and piracy).

Commonwealth (federal) courts:

  • Federal Court – has jurisdiction over 120 plus federal Acts of Parliament.
  • Family Court – jurisdiction over all divorces and maintainence over children and spouses.
  • High Court – primary role is to interpret and enforce the Constitution, amongst functions.

The State Library of NSW provides a useful overview of Australia’s courts and tribunals.

Does this article resonate with you? Please vote below or subscribe to get updates on my future articles

Where to search court records

Most Australian jurisdictions have consolidated their legal records, making the task of searching for a record relatively easy once you know what you are looking for:

JurisdictionCivil or CriminalSourceComments
NSWBothCaseLawGenerally within 24 hours
ACTBothJudgements–
QLDBothCaseLawGenerally within 24 hours
VICBothMultiple WebsitesVaries
SABothJudgements–
TASBothDecisionsPublished on AustLII*
WABothDecisions–
NTBothMultiple Websites–
Federal Court
Family Court
Federal Circuit		BothFederal Law Search–
Federal CourtBothJudgementsReleased within 24 hours
@ForewarnedBlog (2022). Research.

* The Australian Legal Information Institute (AustLII) is jointly operated by the UTS and UNSW law faculties and aims to pubish public legal information, including primary and secondary legal materials. AustLII is not a primary source.

NSW Caselaw advanced search interface
NSW CaseLaw – advanced search interface

Criminal Records are considered ‘sensitive information’ under the Privacy Act

Note that searching court records is different to a National Police Check (‘criminal history check’). Under the Privacy Act 1988 (Cth), an individual’s criminal record is considered a category of sensitive information.

A National Police Check is the appropriate mechanism to understand whether an individual has a criminal record (such as for workforce screening purposes or before contracting with the management team of a prospective business partner). The National Police Check process considers important factors such as Spent Convictions.

Importantly, performing a National Police Check in Australia requires the individual’s informed consent.

How do you search court records?

Public Record checks are typically performed at the early stages of any due diligence or vetting process, once you have a clear understanding of the scope and parties involved. A typical process for searching court records is as follows:

1. Identify the full legal name of all entities and individuals, including close associates and related parties.

2. Determine which databases to query and over what timeframe. The scope and your professional judgement will set the timeframe, whilst jurisdiction is dependent on what you know (or need to know) about the subject. In some cases, a negative search result (i.e. no results returned for a party name) may be all need to know. If you have no idea where they have lived or operated, search every database (you may also need to search overseas).

3. Perform the search(es) and review the results. On the first pass, I use a spreadsheet to manage my searches and put all results in one of three categories: no match, possible match, match. Matches mean there is a record involving your subject (i.e. not another party with the same name). Possible match means you need to spend more time working out whether it’s your subject or not.

4. Assess the implications of your results

Vetting or due diligence is not simply about database checks – anyone can do this. Done properly, background investigations involve identifying potential risks based on what is and is not present (but should be), before determining the implications and what to do about them.

This is where diligence becomes an art. There is nothing in a database to tell you what is missing – this comes down to professional experience, judgement and skill.

Paul Curwell (2022). REfer Chapter 8 in ‘Terrorist Diversion’

5. Identify any other leads which need to be followed up.

6. Update your working papers or case notes, including what you did, when, where and the outcome. Databases and the internet change all the time, so a record that was there five minutes ago may be different when the same search is re-performed.

person working on black laptop
Photo by EVG Kowalievska on Pexels.com

Primary versus Secondary Sources

Wherever possible, primary (original) sources should be used. Secondary source vendors are often more expensive, yet serve two main purposes:

  • For companies that are willing to accept the risk of a record being inaccurate, incomplete, missing or out of date, secondary sources may offer an efficient alternative which enables multiple types of searches to be performed from a single location (e.g. court records, credit ratings, company ownership, land titles) as well as the ability to automating record search and retrieval to your case management system via API.
  • For investigators, secondary sources provide a handy way of quickly identifying potential relationships, transactions or other records which can the be verified via the primary source. Some vendors offer the ability to search all fields in a record, unlike the limited search functionality often offered by primary vendors.

When it comes to secondary, sources, Caveat Emptor: (1) they are not a primary source (hence they could be incomplete or out of date), and (2) they are often a ‘black box’ in terms of search parameters, so you may not actually know what is or is not being searched (some vendors have a nasty habit of changing search functionality without informing their customers, so what worked when you undertook your diligence one week may be completely different the next).

Court Lists

Court lists are published online in most Australian jurisdictions to inform parties to a case when and where they need to be. Often, court lists are published temporarily and subsequently removed. They are not an authoritative source.

Further reading

DISCLAIMER: All information presented on ForewarnedBlog is intended for general information purposes only. The content of ForewarnedBlog should not be considered legal or any other form of advice or opinion on any specific facts or circumstances. Readers should consult their own advisers experts or lawyers on any specific questions they may have. Any reliance placed upon ForewarnedBlog is strictly at the reader’s own risk. The views expressed by the authors are entirely their own and do not represent the views of, nor are they endorsed by, their respective employers. Refer here for full disclaimer.