ECDC COVID-19 landing page    

1 Background

A living systematic review is being carried out on the efficacy, effectiveness and safety of COVID-19 vaccines that are currently, or soon to be, authorised in the EU/EEA. The aim of the review is to provide a living, continuously updated overview on the evidence by specific vaccine product, age groups and SARS-CoV-2 variants.

The review is being conducted by the Robert Koch Institute (RKI), in collaboration with the EU/EEA NITAG Collaboration supported by ECDC. It started on 1 January 2021 and will continue until at least 30 June 2022. Updates are planned on alternate weeks.

The living systematic review is registered in the Prospective Register of Systematic Reviews (PROSPERO; Reg. No. CRD42020208935).

2 Methods

Participants/population
Male and female participants of all age groups are eligible.

Intervention
Any vaccine against COVID-19 which has been approved for use in the EU/EEA. Complete and incomplete dosing schedules will be eligible.

Comparators/control
The comparator will be placebo, no vaccination or a vaccine not directed against COVID-19 (active comparator). In addition, head-to-head trials directly comparing different vaccines against COVID-19 will be included.

Main outcomes
1. Efficacy and effectiveness-related outcomes: SARS-CoV2 infection (PCR-confirmed); COVID-19 symptomatic disease; hospitalisation due to COVID-19 (PCR-confirmed); ICU admission due to COVID-19 (PCR-confirmed); intubation and oxygen supply due to COVID-19 (PCR-confirmed); death due to COVID-19 (PCR-confirmed).
2. Safety-related outcomes: local reactions; systemic events; severe adverse events; enhanced COVID-19 disease; adverse events of special interest (AESI)*. Solicited and unsolicited events will be included.

*An adverse event of special interest (serious or non-serious) is one of scientific and medical concern specific to the sponsor’s product or programme, for which ongoing monitoring and rapid communication by the investigator to the sponsor can be appropriate. Such an event might warrant further investigation in order to characterise and understand it. Depending on the nature of the event, rapid communication by the trial sponsor to other parties (e.g. regulators) might also be warranted.

Measures of effect:
Effect measures for efficacy/effectiveness and safety: relative risk (RR), odds ratio (OR), hazard ratio (HR). These measures will be used to calculate vaccine efficacy/effectiveness (VE, in %) as (1-RR (or OR or HR)) x 100.

Additional outcomes
In addition, complications of COVID-19 (including, but not limited to thromboembolism) and viral shedding will be addressed if reported. Furthermore, other safety outcomes may also be reported if found, as part of the review.

Measures of effect:
Not applicable

Types of study to be included
All phase 2/3, phase 3 and phase 4 studies with designs that have a comparison group are eligible for inclusion. This includes, but is not limited to, randomised controlled trials, cohort studies and case-control studies. For safety data, only phase 2/3 studies, phase 3, phase 4 studies and non-randomised studies with control groups will be considered (including, e.g. self-controlled case series).

Searches
Bi-weekly searches will be conducted in PubMed and Embase and supplemented by hand searches on the preprint servers arXiv, BioRxiv, ChemRxiv, MedRxiv, Preprints.org, ResearchSquare and SSRN and searching the reference lists of included studies and of relevant reviews. No limitations will be made regarding publication dates and languages of publications.

Analysis
Two reviewers will initially screen studies by title and abstract for eligibility. Identified studies will be retrieved in full text and the relevant data will be extracted.

Study characteristics and effect measures will be aggregated in tables. When statistical pooling is appropriate, pooled summary estimates will be calculated with 95% confidence intervals, using fixed-effects and random-effects models. For RCTs, risk of bias will be assessed using the Cochrane risk of bias tool-2 ( RoB-2) and for non-randomized studies, ROBINS-I will be applied.

Quality of the evidence (per outcome) will be assessed using the methodology developed by the Grading of Evidence Assessment Development and Evaluation (GRADE) Working Group.

3 COVID-19 vaccine efficacy and safety results from clinical trials

COVID-19 vaccines licensed for use in the EU/EEA were evaluated in tens of thousands of participants in clinical trials and have met EMA’s scientific standards for safety, efficacy and quality. The vaccines have been shown during clinical trials to be highly effective in providing protection against symptomatic COVID-19 and severe disease.

The EU/EEA authorised COVID-19 vaccines all showed a very good safety profile in clinical trials before receiving approval from the European Medicines Agency (EMA). Since licensing, EMA, other regulatory agencies and international bodies have been continuously monitoring the safety of COVID-19 vaccines. For further information on safety of COVID-19 vaccines please visit EMA.

3.1 Baseline characteristics for efficacy and safety studies included

3.2 Efficacy

3.2.1 Vaxzevria (AstraZeneca AB)

3.2.2 Cominarty (BioNTech Manufacturing GmbH)

3.2.3 COVID-19 Vaccine Janssen (Janssen-Cilag International NV)

3.2.4 Spikevax (Moderna Biotech Spain, S.L.)

3.3 Safety results

3.3.1 Vaxzevria (AstraZeneca AB)



[*] Medical events that have been observed after vaccination, but which are not necessarily related to or caused by the vaccine.

[~] Determining if an event is vaccine-related is made through a number of different methods including observing the rate of an adverse event in the vaccinated population and comparing it with the rate of this event among the unvaccinated population and calculating the background rate of an adverse event in the population and comparing this with the post-vaccination rates of the event. If the background rate of a particular adverse event is not known in a community, this can be compared with the observed rate in the study population with the ‘expected rate’ published by the vaccine regulatory authorities. The use of independent panels of experts are also used to assess if an adverse event may be associated with a vaccine.

3.3.2 Cominarty (BioNTech Manufacturing GmbH)



[^] Pooled results for all ages 16+ years who recevied Comirnaty

[*] Medical events that have been observed after vaccination, but which are not necessarily related to or caused by the vaccine.

[~] Determining if an event is vaccine-related is made through a number of different methods including observing the rate of an adverse event in the vaccinated population and comparing it with the rate of this event among the unvaccinated population and calculating the background rate of an adverse event in the population and comparing this with the post-vaccination rates of the event. If the background rate of a particular adverse event is not known in a community, this can be compared with the observed rate in the study population with the ‘expected rate’ published by the vaccine regulatory authorities. The use of independent panels of experts are also used to assess if an adverse event may be associated with a vaccine.

3.3.3 COVID-19 Vaccine Janssen (Janssen-Cilag International NV)



[*] Medical events that have been observed after vaccination, but which are not necessarily related to or caused by the vaccine.

[~] Determining if an event is vaccine-related is made through a number of different methods including observing the rate of an adverse event in the vaccinated population and comparing it with the rate of this event among the unvaccinated population and calculating the background rate of an adverse event in the population and comparing this with the post-vaccination rates of the event. If the background rate of a particular adverse event is not known in a community, this can be compared with the observed rate in the study population with the ‘expected rate’ published by the vaccine regulatory authorities. The use of independent panels of experts are also used to assess if an adverse event may be associated with a vaccine.

3.3.4 Spikevax (Moderna Biotech Spain, S.L.)



[^] Pooled results for all ages 16+ years who recevied Spikevax

[*] Medical events that have been observed after vaccination, but which are not necessarily related to or caused by the vaccine.

[~] Determining if an event is vaccine-related is made through a number of different methods including observing the rate of an adverse event in the vaccinated population and comparing it with the rate of this event among the unvaccinated population and calculating the background rate of an adverse event in the population and comparing this with the post-vaccination rates of the event. If the background rate of a particular adverse event is not known in a community, this can be compared with the observed rate in the study population with the ‘expected rate’ published by the vaccine regulatory authorities. The use of independent panels of experts are also used to assess if an adverse event may be associated with a vaccine.

3.4 Risk of bias

3.4.1 Vaxzevria (AstraZeneca AB)



[1] Parts of the study population received a different dose of vaccine. Consequences are unclear.

[2] A considerable proportion of randomised participants (both study arms) that was higher than the number of events did not contribute to data analysis. Due to unclear reporting risk bias cannot be completely excluded.

[3] Parts of the study personnel were not blinded. Since data are based on self report (diary) knowledge of group allocation could have influenced reporting.

3.4.2 Cominarty (BioNTech Manufacturing GmbH)



[1] Parts of the study staff were not blinded (e.g. people who administered vaccinations). It was estimated that this had no or only a negligible influence on the risk of bias for this outcome.

[2] A considerable part of the randomised study population (both study arms), which is also significantly larger than the total number of events, was not included in the analysis. The reporting is unclear here, so that a bias cannot be completely ruled out.

[3] Parts of the study personnel were not blinded. Since data are based on self report (diary) knowledge of group allocation could have influenced reporting.

3.4.3 COVID-19 Vaccine Janssen (Janssen-Cilag International NV)



3.4.4 Spikevax (Moderna Biotech Spain, S.L.)



[1] Parts of the study staff were not blinded (e.g. people who administered vaccinations). It was estimated that this had no or only a negligible influence on the risk of bias for this outcome.

[2] A considerable part of the randomised study population (both study arms), which is also significantly larger than the total number of events, was not included in the analysis. The reporting is unclear here, so that a bias cannot be completely ruled out.

[3] Parts of the study staff were not blinded (e.g. people who administered vaccinations). Since these are self-reported events (electronic study diary), possible knowledge of the group membership (knowingly or unknowingly communicated by study staff) could have influenced the reporting or evaluation of events by individual study participants.

3.5 GRADE tables

3.5.1 GRADE Evidence Profile: Vaccination with Vaxzevria (AstraZeneca AB) against COVID-19



[1] Some participants received differing doses of the vaccine; consequences for VE are not completely clear

[2] Residual confounding cannot completely excluded

[3] Study investigated VE after dose 1, therefore indirectness regarding intervention

[4] Part of study personnel was not blinded (incl. vaccine administrators). This could have had an impact on recognition of events/reactions by participants if information on allocation was communicated to them.

[5] Part of control arm received MenACWY vaccine

3.5.2 GRADE Evidence Profile: Vaccination with Cominarty (BioNTech Manufacturing GmbH) against COVID-19



[1] Exclusion of participants in both arms not completely transparently described; impact on results cannot definitely be excluded

[2] Wide 95% confidence interval

[4] Residual confounding cannot be excluded.

[5] Part of study personnel was not blinded (incl. vaccine administrators). This could have had an impact on recognition of events/reactions by participants if information on allocation was communicated to them.

3.5.3 GRADE Evidence Profile: Vaccination with COVID-19 Vaccine Janssen (Janssen-Cilag International NV) against COVID-19



[1] Person-years as denominator

3.5.4 GRADE Evidence Profile: Vaccination with Spikevax (Moderna Biotech Spain, S.L.) against COVID-19



[1] Exclusion of participants in both arms not completely transparently described; impact on results cannot definitely be excluded

[2] Severe COVID-19 used as proxy for hospitalization (indirectness regarding outcome)

[3] Wide 95% confidence interval

3.6 Forest plot

Forest plot of Comirnaty (BNT162b2, BioNTech/Pfizer) and COVID-19 Vaccine Moderna (mRNA-1273) vaccine efficacy against COVID-19 infection in different age groups

4 Effectiveness of COVID-19 vaccines against SARS-CoV-2 infection of any severity with the Delta variant (studies included up to 25 August 2021)

Current evidence shows that COVID-19 vaccines licensed in the EU are moderately effective in preventing SARS-CoV-2 infection with the Delta variant, however vaccine effectiveness against severe disease and hospitalisation remains high.

4.1 Vaccine effectiveness against SARS-CoV-2 infection of any type with the Delta variant

VE estimates against infection of any type (not specified in studies if symptomatic or asymptomatic) ranged between 49% and 82% (for 18-34 years: 90%), with pooled VE 66.9% (95% confidence interval (95%CI): 58.4-73.6%; I²=95.1%) across all studies. VE estimates against asymptomatic infection ranged between 35.9% and 80.2% with pooled VE across all studies 63.1% (95%CI: 40.9-76.9%; I²=93%).

4.1.1 Infection (any type)

4.1.2 Asymptomatic infection

aPreprint; CI, confidence interval; HCW, health care workers; NR, not reported

4.1.3 Forest Plot

4.2 Vaccine effectiveness against SARS-CoV-2 symptomatic infection, severe disease and hospitalisation with the Delta variant

VE against symptomatic infection ranged between 56% and 87.9% with the pooled VE estimate 75.7% (95%CI: 69.3-80.8%; I²=91.9%. VE against severe disease estimates ranged from 81.5% to 100% and the pooled VE was 93.8% (95%CI: 83-98%; I²=0%. VE estimates against hospitalisation ranged between 75% and 96%. Pooled effectiveness against hospitalisation was 90.9% (95%CI: 84.5-94.7%; I²=18.5%).

4.2.1 Symptomatic infection

4.2.2 Severe disease

4.2.3 Hospitalisation

aPreprint; CI, confidence interval; HCW, health care workers; NR, not reported

4.2.4 Forest Plot

4.3 Risk of Bias Assessments

1 adjusted estimates reported, but residual confounding possible; 2 VE not based on sequencing of Delta; 3 at least in in some participants, vaccination status was only self-reported; 4 no confounder-adjusted estimates reported; 5 vaccine (product) not reported; 6 test-negative design