iraqi-dialect-bad-words

iraqi-dialect-bad-words

English | العربية

A profanity filter for the Iraqi Arabic dialect. Detects and masks bad words in user-generated text.

Works in React Native / Expo, React (Vite, Next.js, CRA, …), Vue (Vite, Nuxt) and Node.js, with TypeScript types included. No dependencies.

Install

npm install iraqi-dialect-bad-words
# or
yarn add iraqi-dialect-bad-words
# or
npx expo install iraqi-dialect-bad-words

Usage

import IraqiBadWordsFilter from 'iraqi-dialect-bad-words';
// CommonJS: const { IraqiBadWordsFilter } = require('iraqi-dialect-bad-words');

const filter = new IraqiBadWordsFilter();

filter.isProfane('مرحبا بك');      // false
filter.isProfane('هذا كلام عارة');  // true

filter.clean('هذا كلام عارة');       // 'هذا كلام ****'
filter.clean('هذا كلام عارة', '#');  // 'هذا كلام ####'
filter.clean('هذا كلام عارة', '');       // 'هذا كلام '

React Native example

import { useState } from 'react';
import { TextInput, Text } from 'react-native';
import IraqiBadWordsFilter from 'iraqi-dialect-bad-words';

const filter = new IraqiBadWordsFilter(); // create once, outside the component

export function CommentInput() {
  const [text, setText] = useState('');

  return (
    <>
      <TextInput value={text} onChangeText={setText} />
      {filter.isProfane(text) && <Text>Please keep it respectful.</Text>}
    </>
  );
}

Vue example

Create the filter once in its own module and import it where you need it. Keep it out of ref() and reactive(): the filter never changes, and wrapping it in reactive proxies only slows every check down.

// src/profanity.ts
import IraqiBadWordsFilter from 'iraqi-dialect-bad-words';

export const filter = new IraqiBadWordsFilter();

<script setup lang="ts">
import { ref, computed } from 'vue';
import { filter } from '@/profanity';

const text = ref('');
const isProfane = computed(() => filter.isProfane(text.value));
</script>

<template>
  <textarea v-model="text" dir="auto" />
  <p v-if="isProfane">Please keep it respectful.</p>
  <button :disabled="isProfane">Send</button>
</template>

To highlight bad words, render the text in pieces split at the start and end of each match from inspect() rather than with v-html, so user text can't inject HTML.

In any framework, check the text on your server too (a Nuxt server route or any Node backend can use the same package): a check in the browser can be skipped by calling your API directly.

API

new IraqiBadWordsFilter(options?)

Creates a filter with the built-in word list.

new IraqiBadWordsFilter();                               // profanity only
new IraqiBadWordsFilter({ includeInsults: true });       // also milder insults
new IraqiBadWordsFilter({ includePhrases: true });       // also vulgar phrases
new IraqiBadWordsFilter({ stripSuffixes: true });        // also unlisted suffix forms
new IraqiBadWordsFilter({ dialects: ['mosul'] });        // also Mosul ر → غ forms
new IraqiBadWordsFilter({ customWords: ['word1', 'word2'] });
new IraqiBadWordsFilter({ customWords: [{ word: 'word1', severity: 'mild' }] });
new IraqiBadWordsFilter({ useDefaultList: false, customWords: ['word1'] }); // your words only
new IraqiBadWordsFilter({ safeWords: ['word3'] });        // clean words not to flag
new IraqiBadWordsFilter(['word1', 'word2']);             // same as customWords

Using your own word files

Keep your app's extra words, and clean words you don't want flagged, in JSON files: arrays of single words, the same format as src/safe-stems.json. A word that needs its own severity can be an object instead:

["زفت", { "word": "نذل", "severity": "mild" }]

Import them and pass them as customWords and safeWords. The built-in lists stay on, and your words are added to them:

import IraqiBadWordsFilter from 'iraqi-dialect-bad-words';
import myWords from './my-words.json'; // TypeScript needs "resolveJsonModule": true
import mySafeWords from './my-safe-words.json';
// CommonJS: const myWords = require('./my-words.json');

const filter = new IraqiBadWordsFilter({ customWords: myWords, safeWords: mySafeWords });

Your words go through the same normalization as the built-in lists, so you don't need to list diacritic, ة/ه, stretched or prefixed spellings. Whitespace around a word is trimmed and blank entries are skipped. inspect() reports a match from your word file with source: 'custom', and a match cleared by your safe words with suppressedBy set to that safe word.

A safe word only clears a match that needed a prefix or suffix stripped, like the built-in safe stems. To stop a listed word itself from being flagged, use removeWords().

React Native, Expo, Vite and Next.js bundle the JSON files into your app, so this works the same in each. The filter doesn't read files at runtime. If your lists come from a server, fetch them and pass the arrays to addWords() and addSafeWords().

isProfane(text: string): boolean

Returns true if any word in the text is on the list.

clean(text: string, placeholder = '*'): string

Replaces each bad word with the placeholder, repeated to the word's length. Spaces and punctuation are kept.

The placeholder can also be a function: it gets each match from inspect() and returns the text to put in its place. For example, keep the first letter, or mask only the stronger words:

filter.clean('انت عارة', (match) => {
  const [first, ...rest] = Array.from(match.token);  // Array.from counts 𝐠 or an emoji as one letter
  return first + '*'.repeat(rest.length);
});
// 'انت ع***'

filter.clean(text, (match) => (match.severity === 'mild' ? match.token : '***'));

inspect(text: string, options?): ProfanityMatch[]

Lists each bad word in the text and explains why it matched. Use it to highlight words, to treat words differently by severity (e.g. block 'severe' but only warn on 'mild'), or to let someone review a match before your app blocks it.

filter.inspect('روح للكواد!');
// [{ token: 'للكواد', start: 4, end: 10, normalized: 'لكواد', entry: 'كواد',
//    source: 'words', severity: 'strong', prefix: 'ل', suffix: '', rules: ['stretched', 'prefix'] }]

Pass { includeSuppressed: true } to also see words that matched but were cleared by a safe stem. They are not profane (isProfane and clean ignore them), but they show why a word wasn't flagged:

filter.inspect('وبعيرك', { includeSuppressed: true });
// [{ token: 'وبعيرك', …, entry: 'عيرك', prefix: 'وب', rules: ['prefix'], suppressedBy: 'بعير' }]

rules lists only the rules that made a difference between the typed word and the entry, so a report shows straight away why something matched:

Severity

Every match has a severity, so an app can block some words and only warn on others:

Your own words and phrases are 'strong' unless you give them a severity ({ word: 'نذل', severity: 'mild' }). A word already on a built-in list keeps that list's severity.

const matches = filter.inspect(comment);
if (matches.some((match) => match.severity === 'severe')) rejectComment();
else if (matches.length) flagForReview();

Including the output of inspect in a “Clean word flagged” issue shows exactly which entry matched and which rules it needed; the issue form has a field for it.

addWords(words: (string | CustomWord)[]): void

Adds words to this filter, like the customWords option.

addSafeWords(words: string[]): void / removeSafeWords(words: string[]): void

Adds safe stems to this filter, like the safeWords option, or removes them, including built-in ones (with their Mosul forms when the dialect is on).

removeWords(words: string[]): void

Removes words from this filter, for example a built-in word you don't want flagged in your app.

addPhrases(phrases: (string | CustomPhrase)[]): void / removePhrases(phrases: string[]): void

Adds phrases to this filter, like the customPhrases option, or removes them, including built-in ones.

A phrase is its words separated by spaces. A word can list alternatives separated by |, so one entry covers every combination:

[
  "قوم|كوم بيه|بينو"
]

matches قوم بيه, قوم بينو, كوم بيه and كوم بينو. A phrase matches when its words appear one after another with only spaces or punctuation between them (تعال، العب بيه!). Each word is normalized like any other, so گوم matches the قوم alternative without listing it. The first word can carry an attached prefix (وتعال العب بيه) or the Iraqi progressive د (دقوم بيه). Other forms aren't stripped or guessed, so list each one you want as an alternative.

The whole phrase is reported as one match with the rule phrase, its entry being the line as listed, and clean() masks all of it. removePhrases() takes a line as listed; the order of its alternatives doesn't matter.

The word lists: iraqi-dialect-bad-words/lists

The built-in lists are also exported on their own, for tools that can't use the filter (a database query, a backend in another language):

import { words, insults, phrases, safeStems } from 'iraqi-dialect-bad-words/lists';

words.severe;  // ['عارة', 'كس', …]
words.strong;  // ['كواد', 'كواويد', …]
insults;       // ['كلب', 'حمار', …]

words and phrases are grouped by severity; insults are all 'mild'. The entries are as listed, and they can't be changed. Matching them by plain string comparison catches far less than the filter: it doesn't normalize spelling, strip prefixes, fold Arabizi or add Mosul forms.

What it catches

Matching is done per whole word, after normalizing both the text and the word list, so these all match the same entry:

Because matching is whole-word, a clean word that merely contains a bad word is not flagged.

Split words are found by joining neighbouring pieces when the gap between them is short, on one line, and either has no spaces (عي*ر), sits between single letters (ع ي ر), or has a symbol between pieces of one or two letters (عي * ر). Normal sentences like انا و انت or د. علي stay apart, and the joined word must match an entry exactly. A split safe stem stays safe: ب ع ي ر is بعير (camel).

Known limitations

• Phrases match only as listed, word for word: another word in between (تعال هنا العب بيه) or an unlisted form (بيهم) isn't matched.
• Different words glued together without a space aren't detected. The same word typed several times (عيرعير) is.
• A letter replaced by a symbol (ع*ر, ك*بة) isn't detected: the symbol could stand for any letter, and ع*ر could just as well be عمر.
• Longer pieces split with spaces around the symbol (شرم * وطة) aren't joined, so normal sentences aren't either.
• No word list is complete. Use this as one layer of moderation, not the only one.

Contributing

The word list is the heart of this package, and native speakers are the best judges of it.

• A word is missing: open a “Suggest a word” issue, or add it to src/words.json in a pull request.

• A clean word is flagged: open a “Clean word flagged” issue. If it's flagged only because of a prefix (e.g. بعير "camel" = ب + a bad word), the fix is adding it to src/safe-stems.json. A safe stem written with گ protects only words spelled with گ, so گلب ("heart", Iraqi spelling of قلب) can be safe while كلب is not.

• A sentence is judged wrong: add it to test/fixtures/sentences.json with the verdict a native speaker expects ("profane": true or false, plus "options" if it needs includeInsults, includePhrases or stripSuffixes). Every sentence there runs as a test, so later word-list changes can't silently flip it.

• Clean text gets flagged: add it to test/fixtures/clean-text.txt, which must never produce a match (with or without stripSuffixes, includePhrases and the Mosul dialect), then fix the lists until the tests pass again.

In src/words.json, sexual and explicit words go under "severe" and other profanity under "strong". Words that also have a common, normal meaning (like كلب "dog") belong in src/insults.json, which is opt-in and all 'mild', not in words.json. Phrases of clean words that are vulgar together belong in src/phrases.json, also opt-in.

Development

npm install
npm test           # run tests, including the word-list checks and the clean-text check
npm run typecheck  # type-check
npm run build      # build dist/
npm run size       # check the built files are within their size budgets
npm run smoke      # load the built package with plain Node, as CI does on Node 14-18
npm run check -- article.txt --phrases   # list every match in a text file

npm run check runs the filter over a text file (or stdin, e.g. pbpaste | npm run check) and prints each match with its line and column, the entry it matched, its source and rules, and the text around it. Flags turn on options: --insults, --phrases, --suffixes, --mosul, and --clean to also print the masked text. Running it on articles you know are clean is the quickest way to find false positives: each match is either a missing safe stem or a word that belongs in an opt-in list.

npm test also checks the lists themselves: no entry may normalize to the same word as another (گحبة and قحبة, qahba and gahba), no entry may be one the filter already catches without it (الكس is ال + كس), and every safe stem must clear a match. When a new entry fails, the message names the entry it duplicates and the rules that already catch it.

README.ar.md is an Arabic translation of this file. When you change one, change the other too.

License

MIT