Identifying efflux proteins with PSSMs and a multi-window CNNIdentifying efflux proteins with PSSMs and a multi-window CNNIdentifying efflux proteins with PSSMs and a multi-window CNN

The research question

The World Health Organization (WHO) describes antimicrobial resistance as a major global health threat, and estimates that bacterial antimicrobial resistance was associated with more than 4.7 million deaths worldwide in 2021.¹

An efflux protein is a transport protein embedded in the cell membrane that moves toxic substances out of the cell, antibiotics among them. A 2023 review describes antibiotic efflux as one of the most common resistance mechanisms across pathogenic bacteria.²

Confirming that a membrane protein is an efflux protein normally takes lab experiments, and experiments on membrane proteins are slow and costly. Screening sequences computationally first lets those experiments focus on the most promising candidates. The project split the problem into two binary classification tasks:

• Efflux vs. membrane proteins: find efflux proteins among membrane proteins in general.
• Efflux vs. transport proteins: find efflux proteins among transport proteins. This one is harder, since efflux proteins are themselves a kind of transport protein.

Protein sequences: the input

A protein is a long chain of amino acids joined one after another. Living things commonly use 20 amino acids, and each has a one-letter code, such as M for methionine, K for lysine and A for alanine. Writing out a protein’s amino acids in order, from start to end, gives its amino-acid sequence, such as MKTLLIAVAG….

Proteins range from a few dozen to thousands of amino acids long. The chain folds into a particular 3D structure, and the structure determines what the protein does, so without lab experiments the sequence is the most direct clue to a protein’s function, and it is the model’s only input.

Amino-acid sequenceMKTLLIAVAGFLLSAWQPERKDGS

HydrophobicPolarPositive chargeNegative charge

Folded (illustration)

One-hot and PSSM: feature engineering

A model works on numbers, so each position in the sequence first becomes a row of numbers. For a sequence of length L, both encodings produce an L × 20 matrix with one column per amino acid; they differ in what goes in the cells.

One-hot encoding is the most direct option: each row has a 1 for the amino acid at that position and 0 in the other 19 cells. It records only what is there, so any two different amino acids are equally unalike.

A position-specific scoring matrix (PSSM) starts from an observation in biology: proteins with similar functions have similar sequences, but not identical ones. They usually share a common ancestor. Over evolution, positions that matter for the function rarely change, while other positions can switch to a different amino acid and the protein still works.

So instead of looking at a single sequence, it helps to line up a group of similar sequences and ask, for each position, what is usually there and what else can be there. A PSSM is that summary. It is built by searching a large protein database repeatedly with PSI-BLAST: each round combines the significant alignments it found into a position-specific score matrix, which is then used for the next round.³ Each cell of the result is a score for how strongly that amino acid tends to appear at that position across the related proteins:

• Conserved positions: almost every related sequence has the same amino acid here, so only one or two amino acids score high and the rest score negative. Such positions are usually important to the function.
• Variable positions: different sequences put different amino acids here, so many amino acids score about the same.

In other words, one-hot tells the model what this one sequence looks like; a PSSM tells it what this kind of protein usually looks like, and where it can vary.

Click a letter in the sequence to compare the same position under both encodings:

Similar sequences

1MKTLLIAVAGFLLSAWQPERKDGS

2LRSILIAVANYMVTAWQPERREST

3MKTVFIAVADLILGAWEPEKNNGK

4VQNLIIGVAGFVINAWQPDREDAS

5IRAMVVAVSSWLFSAWQPERKSND

6MKSFLIAVAEIALAAWQPERTGGA

SeqMKTLLIAVAGFLLSAWQPERKDGS

A

R

N

D

C

Q

E

G

H

I

L

K

M

F

P

S

T

W

Y

V

One-hot: at each position only the amino acid that is there is 1; the rest are 0.PSSM: conserved columns have one or two high scores; variable columns are spread out.

Value01

Score−50+9

MKTLLIAVAGFLLSAWQPERKDGS

A

R

N

D

C

Q

E

G

H

I

L

K

M

F

P

S

T

W

Y

V

PSSM: conserved columns have one or two high scores; variable columns are spread out.

Score−50+9

Multi-window CNN: the classifier

With PSSMs in hand, I needed a model to read an L × 20 matrix. I chose a convolutional neural network (CNN), because it is good at finding local patterns in long sequences. Convolution along a sequence works like this:

1. Each filter has a fixed width w: it is a window that covers the 20 scores of w consecutive positions.
2. The window slides along the sequence one position at a time. At each position, the scores inside the window are multiplied by the filter’s weights and summed into one score. A high score means that segment matches the pattern the filter has learned.
3. The model learns many filters at once, each for a different pattern.

I started with a single window width, and found that whichever width I picked, some cases did poorly. My reading was that meaningful segments of a protein vary in length, and one width only catches one length. So I followed the multi-window CNN (mCNN) approach and scanned the same PSSM with several window widths at once, then combined the results.⁴ To choose the widths, I first tested each width on its own, then compared combinations of several widths.

The next problem was that sequences differ in length. So that all sequences could go through the model together, I zero-padded every PSSM to the same length. After the scan, each filter produces a whole row of scores along the sequence. I used one-max pooling to keep only each filter’s highest score, so each filter outputs a single number wherever its pattern appears in the sequence, and all the filters together give a feature vector of fixed length. Finally, after a dropout layer, I added a fully connected layer with 1,024 neurons and a softmax to output whether the protein is an efflux protein.

Pick window widths, press play to watch the scan, then try making the sequence longer:

MKTLLIAVAGFLLSAWQPERKDGS

A

R

N

D

C

Q

E

G

H

I

L

K

M

F

P

S

T

W

Y

V

4

8

12

Pos16121824

Feature vector · length 13983101Fixed length, whatever the sequence length

The width-8 window scores highest at positions 15–22, right over a conserved region.

Putting these steps together gives the whole model:

01

Input · L × 20, L = 24

MKTLLIAVAGFLLSAWQPERKDGS

A

R

N

D

C

Q

E

G

H

I

L

K

M

F

P

S

T

W

Y

V

02

Convolution · Three filters per width; the bars show one of them

4

8

12

03

One-max pooling

439

883

12101

04

Concatenation · Length = widths × filters per width = 2 × 3 = 6

Width 4, filter 1: 39Width 4, filter 2: 35Width 4, filter 3: 28Width 8, filter 1: 83Width 8, filter 2: 65Width 8, filter 3: 52Width 12, filter 1: 101Width 12, filter 2: 83Width 12, filter 3: 74

Fixed length, whatever the sequence length

05

Dropout (0.7) → fully connected layer (1,024 neurons) → softmax · Illustrative

⋯

Efflux92%

Not efflux8%

During training, dropout switches off a random 70% of the features each time, to reduce overfitting; prediction uses all of them.

A fully connected layer combines the features, and softmax gives the probability of being an efflux protein.

Evaluation: MCC

Efflux proteins are far fewer than the other two classes, only about a tenth of the membrane dataset. On data this imbalanced, accuracy alone is misleading: a model that calls every protein “not efflux” would still be over 90% accurate while being useless.

The project therefore evaluated models mainly with MCC (the Matthews correlation coefficient). MCC accounts for errors on both classes and ranges from −1 to 1: 1 means every prediction is right, 0 means no better than random guessing, and the “call everything not efflux” model above scores 0.

Results

The tables below come from different rounds of experiments with different window combinations and filter counts, so the PSSM + mCNN MCC values are not all the same; the final model’s results are in the last table.

Data and setup

• Source: membrane, transport and efflux protein sequences from UniProt, with sequences shorter than 50 amino acids removed.
• Removing near-duplicates: CD-HIT grouped sequences more than 70% similar and kept one per group, so nearly identical sequences would not appear in both training and test data.
• Split: each class was split 80% / 20% into training data and an independent test set.
• Imbalance: SMOTE was applied to the training data to add samples to the minority class.
• Tuning: window combinations and filter counts were chosen by K-fold cross-validation.

Features: PSSMs beat one-hot

With the same mCNN model:

Classifier: several widths beat one

For efflux vs. membrane:

Going from one width to two raised MCC from 0.85 to 0.96; with more widths, MCC stayed between 0.93 and 0.96 with no clear further gain, so the combination of widths is a hyperparameter to tune. When filter counts were tested with four widths (28, 16, 24, 20), raising the number of filters from 4 to 256 also raised MCC from 0.65 to 0.98.

Against conventional classifiers

On the independent test set, with the same PSSM features:

Conclusion

A PSSM brings evolutionary information into every position: proteins with similar functions have similar but not identical sequences, so the model can tell which positions are conserved. A multi-window CNN scans with several window widths at once and uses one-max pooling to reduce each filter’s scores to a single number. Together they separated efflux proteins from membrane and transport proteins well on this dataset.

Looking back, the project also had some limitations:

• One dataset only: every result comes from the same data and split. An MCC near 0.99 is very high; to confirm the results carry over to other data, they could be checked on an independently collected dataset.
• Similarity threshold: a 70% threshold removes only very close sequences, so moderately similar sequences may still sit on both sides of the split. For a stricter evaluation, a lower threshold could be considered.
• Cost of PSSMs: every sequence needs repeated PSI-BLAST searches against a large database, which becomes slow for many sequences. This is one reason later work moved toward protein language models.⁵
• Only the first step: the model screens for likely efflux proteins; structure analysis and drug screening afterwards were outside the project’s scope.

Notes

1. World Health Organization, Antimicrobial resistance (fact sheet).

2. Gaurav, A. et al. (2023). Role of bacterial efflux pumps in antibiotic resistance, virulence, and strategies to discover novel efflux pump inhibitors. Microbiology, 169(5), 001333.

3. Altschul, S. F. et al. (1997). Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Research, 25(17), 3389–3402.

4. The multi-window scanning approach follows Ho, Q.-T., Le, N. Q. K., & Ou, Y.-Y. (2022). mCNN-ETC: identifying electron transporters and their functional families by using multiple windows scanning techniques in convolutional neural networks with evolutionary information of protein sequences. Briefings in Bioinformatics, 23(1), bbab352.

5. For the shift from multiple sequence alignment to protein language models, see the survey our lab later published, which traces features for membrane-protein sequences from evolutionary information such as PSSMs to pre-trained language models: Ou, Y.-Y., Ho, Q.-T., & Chang, H.-T. (2023). Recent advances in features generation for membrane protein sequences: From multiple sequence alignment to pre-trained language models. Proteomics, 23(23–24), 2200494.