Author: Jason Hine, MD FAAEM
Originally Published: Common Sense May/June 2017
Emergency medicine is a relatively young specialty. Compared to internal medicine and surgery, the two pillars of medicine, it is essentially a toddler. But with this youthfulness comes an advantage. It has allowed us to be on the forefront of medical education’s evolution. We have not been bound by traditions or volumes of ancient leather-bound texts handed down through generations. Our forefathers blazed a new trail in medicine and took with them many fresh thinkers and visionaries.
We have some of the most innovative and cutting-edge education of all medical specialties. We have pioneered new ways of teaching, actively seeking out and utilizing technologies as they develop. The most prominent has been FOAM, or free open-access medical education. This once small resource has grown to massive size. With growth, however, come challenges. While the ease of contribution has driven FOAM’s success, it has the downside of unregulated and unchecked production.
By its very nature, FOAM gives everyone a voice. This culture of sharing, comparing, and questioning is the machinery through which we are able to propose new ways of thinking, challenge the old, and make progress together. While unimpeded content production has been a part of FOAM since its advent, the explosion of original contributions over the last several years has caused this to be seen as a problem. Back when EM:RAP*, EMCrit, and ERCast were the only big content producers around, it was easier to see you were getting well researched and up to date information from a credible educator. Today’s trainee researching a topic on FOAM still has these titans to learn from, but is also presented with hundreds of other podcasts and blogs, all without any external validation.
This needs to change. We need to lay the groundwork for vetting FOAM. Scientific journals have used peer review as one means of quality assurance for decades. While criticism of this system exists, it provides a place to start. In this system, an author produces content and submits it to a journal, which has its own team of peers who review the content for quality and relevance.1 If the piece gets approval, it is published. There are certainly pros and cons to this formula. Pros include its formulaic nature, the ability to utilize the expertise of relevant authorities in the field and, if functioning in its purest form, the ability to prevent erroneous or irrelevant content from being published. Cons include the biased selection of peers, the journal’s association with content which can lead to publication bias, and the huge number of journals which leads to decentralization of information. While this system can serve as a foundation on which to build, it should be stated clearly that FOAM is not scientific research. Its purpose is not the same and therefore it should not undergo the same validation process.
Some have attempted to re-imagine peer review for the FOAM world. Most notably, Academic Life in Emergency Medicine (ALiEM) has created the Approved Instructional Resources (AIR) series, through which their team assesses high-profile FOAM resources. In their system, the board of directors first evaluates blogs and podcasts by a Social Media Index (SMI) score. The SMI gauges the source’s amount of exposure in the FOAM community. From there, the content of the resource is assessed and scored in various attributes including accuracy, utility, evidence base, and referencing.2 Approved resources receive the AIR stamp of approval and are included in ALiEM’s Individualized Interactive Instruction (III) initiative.3
This framework is ALiEM’s approach and, like peer review, has its pros and cons. An in-depth look at these would require an entire article. One immediate criticism is in the SMI score. We certainly know that social media popularity does not assure quality. In our current approach to FOAM we rely on the notion that the cream will rise, but one must believe wholly in this idea to adopt the SMI score into the vetting process. Doing so erects a barrier to the talented but as yet undiscovered. On the other hand, in an expanding sea of original productions, it provides some semblance of a selection process.
With this foundation one can begin to see the system by which FOAM could be globally vetted. Using structures already in place, we have access to a robust and unbiased group of peers. In the modern digital era the analysis of a resource can happen asynchronously across states, regions, and the world. What we currently lack is the organization, collaboration, and unity to do it. To be successful, the vetting process will have to be reviewed and accepted by multiple national societies and communities. Open communication and partnership will be vital. If we can do this, our specialty can create a vetting process for FOAM and bring order to this fantastic, powerful resource.
Have an opinion, commentary, or perspective on the matter? Interested in helping bring this infrastructure to fruition? Please reach out to me via Twitter: @JayHineMD
* Notably not FOAM.
References:
1. http://undsci.berkeley.edu/article/howscienceworks_16
2. SMI index: https://www.aliem.com/social-media-index/
3. ALIEM AIR: https://www.aliemu.com/air/