Sunday, February 9, 2014

Mooconomics

An Economist article and Alex Tabarrok in a Marginal Revolution blog post weigh in on the future and economic structure of moocs (massively open online classes).

A few thoughts on this question, based on my experience teaching a Coursera mooc last fall. (This was supposed to be a short post, and grew out of control. Oh well.)

Yes, moocs potentially upend the fundamental economic structure of teaching. Teaching had been a high marginal cost business. Moocs are a nearly zero marginal cost business.

But.. For now, Moocs are a quite high fixed-cost business. Putting a class up in a mooc is not quite as much work as writing a textbook, but it's nowhere near as easy as teaching a new class. If you're tempted, beware!  Preparing, taping, editing and uploading a lecture is not the same as walking in, telling a few jokes, and getting through the week. Fixing anything that went wrong or updating is costly too.

Part of the high fixed costs lie in the limitations of the software. This is still version 1.0 stuff. Quizzes and assessments are key to the success of a class, and these are still particularly rudimentary. On Coursera, really, not much more than multiple choice and numerical entry works well.  Coursera software does not allow parts to questions, or students to try question 1, get a hint, solve it, see the answer,  and then go on to question 2 which builds on question 1. You can't even cut and paste pictures. As an instructor, taking those "prove x theorem" or "how do you resolve y puzzle" problems and turning them into meaningful multiple choice or numeric entry questions is the hardest part. Artificial intelligence programs reading text entries, guiding the student to different material based on his misunderstandings, and so on... this is all personal flying jet packs dreams.

The difficulty of using the software and its capabilities should get better. Version 1.0 turns to version 2.0 (And then becomes bloatware, and we all switch to a new platform and recode everything...) But if you prepare a mooc, do not dream that you are putting it in stone for generations. You're an early adopter, and you're going to be updating and redoing it for better and different software for a long time, paying fixed costs year after year.

Don't dream of doing a mooc on your own. You need video and IT help. Most of all, you need pedagogical help, people who keep up with the fast-evolving art of how to successfully port classes on moocs. I had that help at the University of Chicago, and it saved me from horrible beginner blunders. Example: I wanted to tape my live classes. No, Emily, who was in charge of my class, insisted that we do it months ahead of time in 5-8 minute segments. This little tip alone saved me from what the Financial Times labeled, in its review of Wharton's moocs, "the death march of moocs," so boring you will have "blood pouring out of your eyes" -- and the necessity to do it all over again right.

High and continuing fixed costs has economic and strategic lessons. University administrators hot to "go online," who figure they'll just get a few volunteers to throw their popular classes at the internet need to face the fact that creating high quality online classes is going to cost real money -- money for IT, video, and mooc specialist support staff; and most of all faculty, who are not going to do this for free.

Yet, for the moment, the market price is "free." That leads to a bit of conundrum -- big expensive ongoing fixed costs to produce something that we give away? How will we "monetize" it? What will the economic model be, and how will moocs change the higher education market?

The grumpy response to moocs: When Gutenberg invented moveable type, universities reacted in horror. "They'll just read the textbook. Nobody will come to lectures anymore!" It didn't happen. Why should we worry now?

As Alex pointed out, there is a good analogy between textbook publishing and mooc creation -- high fixed, low marginal cost (now zero for textbooks too). It leads to superstars with established brand names taking over the market, and Alex speculated that publishers will know how to recover costs.

A lot of mooc is, in fact, a modern textbook -- because the twitter generation does not read. Forcing my campus students  to watch the lecture videos and answer some simple quiz questions, covering the basic expository material, before coming to class -- all checked and graded electronically -- worked wonders to produce well prepared students and a brilliant level of discussion. Several students commented that the video lectures were better than the real thing, because they could stop and rewind as necessary. The "flipped classroom" model works.

The "flipped classroom" component will, I think, will be a very important use for online tools in business education especially. Our classes are infrequent -- once a week for three hours at best. Our international program meets twice for 5 days in a row, three hours per day. We also offer intensive 1-3 day custom programs. An experience consisting of a strong online component, in which students work a little bit each day over a long period, mixed with online connection through forums, videos, chats, etc., all as background for classroom interaction -- which also cements relationships formed online -- can be a big improvement on what we do now.

There is a lot of worry about moocs substituting for campus experience. At top universities, at least, that seems unlikely to me. The online version is the foundation for the campus version. Well done, the campus class gets a layer of depth, practice in application, additional material, that the mooc cannot offer.

But a warning to faculty: Teaching the flipped classroom is a lot harder! The old model, we pretend to teach, you pretend to learn, filling the board with equations or droning on for an hour and a half, is really easy compared to guiding a good discussion or working on some problems together.

This thought suggests another model. Better pedagogy will demand online components to standard teaching, and that will demand the fixed costs be incurred -- or bought from someone else. Once online, the zero marginal cost principle means it's costless to open it to the rest of the world.

So, what does the classroom experience, on campus, in a degree-granting institution, produce that "read the textbook" does not produce -- and were on that spectrum will the mooc lie?

- Social interaction, and commitment. An early lesson of online learning is the huge dropout rate -- about the same as "I'll just read the book." When the class has a "session," online forums, a sense of community and interaction, people keep going.

The importance of the social component was, to me, another lesson of my mooc. But the social media part of moocs are also in their infancy. We used Coursera's forums, and Google hangouts. (Another kludge, not really ready for this use.) The next round of mooc software and its usage has to really develop the social media and interaction component, above the video lecture and multiple choice quiz component.

- How to do it wrong. Most of my skill as a classroom teacher comes from the fact that I know most of the wrong answers as well as the right ones. When students mistake expected return for actual return, foreknowledge for probability distribution, variance for downside risk, I can spot that quickly.

- Application. Real knowledge come from being able to use and apply ideas. Multiple choice tests can't really push you there.

On these dimensions, online is about halfway. The forums, google chats, and growing community between students are not as good as a high quality classroom experience. They are much better than a mediocre class at a university in the middle of nowhere.

What kinds of classes work well online? Surely, the kinds where "read the book" worked are the easiest, where the material at hand is clear and settled, and teaching has moved to the realm of pedagogical knowledge. Introductory math or science classes strike me as good examples, as well as practical classes -- how to pass your pilots' license written test. In these classes, not only is "how to do it right" quite settled, but there is a codified knowledge about all the ways to do it wrong, that can be probed with multiple choice testing.

Many Coursera classes seem to appeal to an "adult education" market wanting fairly simple introductory courses. There, snazzy lectures are important, and my struggles with machine-gradeable quizzes less so.

About 4000 people watched my videos all the way through, and about 300 did all the work -- reporting about 16 hours a week on average. In the current market, there is a strong demand for the former sort of "taste" class, and less demand for a do-all-the-work PhD class. I'm still pretty amazed that 300 people turned out for the latter, but as I revise the course I'll make the "taste" version more accessible.  People who work cannot take 16 hours a week to do problems.  It seems pretty clear that, absent a big push to bundle moocs into something representing a degree, they have to be broken down to smaller chunks.

Moving up the scale, there is (I hope) a reason for the existence of a research university, that there is an important externality between teaching and research. Researchers benefit from having to sift through, clarify, and simplify new ideas. Students benefit, in that people who are actively doing research really understand the last 40 -50 years of knowledge accumulation and teach it a lot better. Even the introductory finance classes at Booth are taught well past the available textbooks, reflecting a revised understanding of such basics as what the CAPM and portfolio theory are as well as the subtleties of more recent research. High school classes do not need researchers, they need pedagogical experts.

Here, the role of the online class seems murkier, and it will be interesting to see if they can move past the introductory classes which now dominate. For faculty, the low cost of innovating a class, and updating class materials -- at least compared to putting it online or writing a textbook -- is key. Better software that lowers these costs is going to be vital. For students, though, the interaction, the thinking through puzzles and wrong ways of doing things, are more important.

On the other hand, moocs may be a boon to specialization. Already, I received a few inquiries from faculty at other institutions who wanted to use my mooc as part of their program. They have small PhD programs, or don't have faculty in my area, so they can't staff a full class on asset pricing theory. So, use my mooc; guided by a TA or faculty member who isn't expert in the area, and isn't paid a full teaching credit. So much knowledge is so specialized, and no university can host an expert in anything. There has been some bemoaning the "economics of superstar teachers" that moocs may create. For freshman physics, maybe. But if there is one world superstar in, say, second century sanskrit poetry, (I'm making this up), now every university in the world can offer a class in second-century sanskrit poetry.

However, no question about it, the deadly boring hour and a half lecture in a hall with 100 people by a mediocre professor teaching utterly standard material is just dead, RIP.  And universities and classes which offer nothing more to their campus students will indeed be pressed.

Moocs represent one of the few places in which people are actually studying pedagogy at universities, and trying to improve it. Technology facilitates data collection. Unless you're in the business, you would be very surprised to learn that college professors receive essentially no training in how to teach, no supervision of their classroom activities, and little feedback beyond a numerical rating at the end of the quarter. Though we are data-oriented empirical social scientists in our research lives, we do essentially no measurement or study of teaching effectiveness. The data collection and analysis that moocs provide may change that. Also, the presence (and necessity!) of a mooc staff means there are people whose job it is to keep up with those pedagogical lessons.

All this is in its infancy. As with all else mooc, the hype is well beyond the reality. Coursera only hosts simple analytics. For anything else, you just get a huge unsorted csv file. But the potential is there.

Business model. 

So now, back to who pays and how do we pay the large fixed costs for all this zero marginal cost free stuff.

Universities offer more than classes. They offer student selection, curriculum structure, branding, community, and certification. Business schools are often accused that that's all we do! Important social connections form at schools, and schools form powerful alumni networks. Schools have job placement departments.

There are thousands of mooc classes out there. There is certainly room for an institution that has searched for the good ones, and says "we offer a degree if you make it through mooc x y and z."  If the university offers guided study groups, all the better. Universities are loath to offer real course credit for moocs. So there still is a business model much like the textbook model: universities charge students for degrees. And moocs can charge universities for use of their materials in this case, while allowing them free otherwise. This just extends the model many places have, that course materials are freely available. (MIT was a pioneer here. All my class materials are available too, here.)

Another model: Some bands put their music on the internet for free, which raises their concert revenue. Most universities are deadly serious about "brand" and "global presence."  They are hungry to turn alumni and friends into a "community" of supporters, and develop "lifelong" interactions. If an alum is looking for a job, or trying to land a contract, it is awfully nice if the potential employer or partner says "you went to Chicago? Wow you must be smart." Look at the glossy magazines that come in your mail, or the Booth school's excellent Capital Ideas, University of Chicago News or our Centers in Beijing Delhi and Paris or the Alumni and Development budget at any university. This stuff costs real money. And it's a star system. There is only room for a few names at the top.

In this race, distinctive high quality online classes could be an effective tool. Instead of reading a magazine article or website about "all the great stuff going on at Chicago," imagine if ten thousand people a year take distinctive well-branded Chicago online classes. Imagine further that they meet "real" students and become part of the "community." That could be a potent tool in this race, and worth the costs involved.

Our business school  has for a long time wanted to connect with alumni -- and have alumni connect with each other -- on an intellectual level, not just gee-whiz, hire our students, give us money. Online versions of the distinctive and new classes can do that.

In my exit survey, I asked how taking this class changed the students' perceptions of the University of Chicago or the Booth school. About half said their impression was improved, and about half said no change because they already have a good impression.

In sum, one "model" that may work is that online classes, expensive though they are to produce, are simply one of the many fixed costs that universities anxious for widespread "brand recognition" must undertake, without direct "monetization," just as they hire star research faculty.

This vision dictates a different approach however. For the moment most online classes have been simplified versions of introductory classes. Introductory classes are a dime a dozen, and not "distinctive" or "brand promoting." That objective will have to lead to universities putting their most distinctive, high-level, and popular classes online.

Already, the top universities have given up on language instruction online. This is a great example of a field that makes a lot of sense online, but in which expertise in pedagogy is more important than research expertise in the subject matter.

This "community" thing is interesting. I saw quite a few students form connections through the forum on my class. A group of students came from a skype group from all over the world that takes online classes together. Alas, the social network that formed on the class focus group disappeared with the class. Message to administrators, you want to keep these communities alive.

The marketing, branding, and selection thing goes both ways. Simply posting a class on Coursera, you attract the sort of people who are browsing Coursera classes. They fit the typical "adult education" model well. Of 37000 who might browse a course in asset pricing, not many make it through the first lecture, "review of stochastic calculus." You want to connect to the people who might be interested in the class, who maybe aren't taking a lot of online classes already. Direct marketing, communication, using the network of engaged alumni or other university or brand specific networks will emerge as being important, rather than pooling every student in the world, every class in the world, in one big bazaar.  Similarly,  measuring success in this model has to go past the seduction of click counts. You have to identify a target audience and see if you're connecting with them.

Those of us in business schools tend to be revenue-minded. We should remember that the rest of the university does not really live off tuition revenues. Universities are often described as a hedge fund that offers some classes, or in some cases a football team.  There is some truth to this. We don't try to "monetize" every single one of our research efforts. In fact, the investments that top universities make in research, labs, and super star professors, who give away the fruit of that research in academic journals, far exceeds conceivable investments in online efforts. We are, in the end, nonprofit institutions that give away what used to be called knowledge and is now called intellectual property.

In sum, I see two possible paths this could take. Moocs could become snazzy textbooks, "monetized" by textbook companies. Or they could become a central way universities establish a presence beyond local campus walls, generating the "superstar" market. Or, perhaps, a bit of both.

Update: The Economist Free Exchange blog though not free offers some interesting thoughts, especially on the survival of second-tier universities. It's still underrating the social component though, I think. Otherwise reading the textbook and taking a standardized test would have already displaced those universities.

13 comments:

  1. Professor Cochrane, as a recent graduate(?) of your coursera course, I feel I might actually be qualified to comment (rare!).

    First: 'There is certainly room for an institution that has searched for the good ones, and says "we offer a degree if you make it through mooc x y and z."' Coursera is now offering paid 'specialisations'. It's not quite the same as a degree, granted. But its perhaps somewhere between the current Statement of Accomplishment and a degree. That said, it doesn't appeal to me at all.

    'Several students commented that the video lectures were better than the real thing, because they could stop and rewind as necessary.' I agree! I listened to the lectures on my run into work as a first pass - a get-the-idea-of-where-this-is-going listen - and then I would sit down and watch (and note) the lectures over Monday-Wednesday. This is actually a great way to learn; though, granted, probably not for everyone.

    I too am amazed 300 completed the coursework; you lived up to your promise, the course was hard! I have to disagree with: 'People who work cannot take 16 hours a week to do problems' - I did, and I'm sure at least some of the other 300 did as well. It was _hard_, but it was doable. And, for shorter courses, a quick sprint is feasible - I don't disagree with your comment about breaking them down into shorter chunks (your course was about as long as I could manage, I was beginning to run out of steam). That said, I took a day off work to complete the final, so maybe that invalidates my argument.

    On the assignments: I see this as the biggest stumbling block to MOOC growth. Your assignments did a pretty good job of getting around the limitations, but they are nonetheless very tight limitations. There were a few econometric-based questions that I gave up on because I couldn't get coursera to take the numbers. I think, in most cases, I was fairly close, but may have been doing something stupid. But, there were occasions where I took the zero for 'no work' over the zero for 'a lot of work but not quite there' because I didn't have the time. And multiple choice questions are obviously less than ideal but still nonetheless (brute force, obvious answers etc etc). I don't want to sound as if I'm detracting from your course here; the assignments were still excellent and _really_ helped me. The algebra questions worked very well, I never had any problems with machine grading of those. But, online assignments are limiting, is my point. I imagine those taking the course at booth got an even better course!

    Lastly, thanks (again!). You are not kidding about the high fixed costs. I've followed a number of coursera courses - although this is the first I've done the assignments for - and this is one of the highest-production-quality courses I've taken (my only gripe, the intro music. It was a good intro, but after what must have been close to five hundred listens - start and end, 100 lectures, probably average of twice each lecture - it wore thin...).

    ReplyDelete
  2. I read both this post as well as the internal paper that Pr Gibbs had posted online with great interest. Having been skeptical of the hype surrounding MOOCS it is very interesting to read about actual observations.

    The Flipped Classroom is potentially a very exciting idea, even if daunting and more difficult for the professor. Putting the analytics to even better use, you can see which concepts are well understood by the class, and which need more work. This could mean that each in-class session is tailored to the needs of that class on a given day.

    "Knowing the wrong answers" can be a path to improving the software. I could see linking questions to time slots in the videos. When you get your results, you don't just get a 7/10, but you also get links to passages where the concept you missed is explained, so you can review that section again.

    In the same vein, I could see not just answer key explanations of why the right answer is right, but also explanations of what common mistakes lead to choosing wrong answers, and bringing them on the path to better understanding the concept.

    Standalone, I didn't see much value in MOOCS. Under the "improved interactive textbook" understanding I do believe they can greatly contribute to the in class learning experience.

    ReplyDelete
  3. I also completed Prof. Cocharne Coursera course, and I have to agree with Angus. It is customary for CFA candidates to do 15h a week for 18 weeks. May I suggest that most students who are interested by a PhD-level 'MOOC' want more and not less of the stuff. I believe the assignments were quite good and you have to accept that on-line delivery will work better in some aspects and worst in others in comparison to a live class.

    Now regarding business models, may I suggest that for some time the successful MOOCs as provided by Universities will likely be akin to the WSJ, FT and NYT web sites - i.e., an equilibrium between freebies and the full kabang. The freebies (the MOOCs) are of the same quality as the full kabang or indicative of the quality of full kabang (i.e. on-site curriculum degrees), but only allow for 'partial knowledge'. It is an effective way for top universities to consolidate their brand and will not be easily matched by 'a university in the middle of nowhere'. I assume Prof. Cocharne was not referring to a certain university in up-state NY - some people can be sensitive ;)

    ReplyDelete
  4. I took half your course on Asset Pricing on Coursera before I dropped out (work pressures meant I wasn't able to keep up with the assignments, and once I had missed a couple of assignments I had no incentive to keep going in the course). Despite the technical issues you've mentioned here, I must say I absolutely loved and enjoyed the assignments, and must say that they were the main means of learning - the book and your lectures were secondary! So kudos on that!

    And what I absolutely loved about your course was about how you brought together several concepts I'd learnt in different courses when I'd been to business school (at one of the better schools in India), and tied them all together. I hope to take your course again when you offer it next year (I hope you will) and take it forward from where I left off this year.

    And here is a blog post I'd written halfway through your course, when I was still diligently doing my assignments:

    http://noenthuda.com/blog/2013/10/07/studying-on-coursera/

    ReplyDelete
  5. No need to caveat the length of your post. I find the details of your MOOC experience to be fascinating, and the large number of your anecdotal observations paints a richer picture than would a post half the length. Online education is a very important subject, yet much of what I've read on this topic looks like it was written by someone pushing an agenda. Thank you for furthering my knowledge in this area.

    ReplyDelete
  6. Thought this was great-hadn't considered how difficult it is today to teach a course in a MOOC vs on-campus but it makes sense. I guess similarly it will make sense that as the tools get better and professors move up the learning curve the barriers to adding new MOOCs will decrease.

    ReplyDelete
  7. It's very helpful to read an honest and thoughtful report from someone who has delivered a Coursera MOOC and can reflect on it critically (theory without practice makes more sense in theory than in practice). I think you are right about the importance of the social aspect of learning. So much goes on in a face-to-face engagement, especially in small groups, that we simply don't understand very well. And we are a long way from designing spaces and tools that can support the richness of social interaction online. One thing that current Coursera MOOCs don't do well is to support social networked learning. The exchanges are still predominantly between the teacher and the student; it's still a one-to-many, broadcast model. The potential of many-to-many social interactions, crowdsourcing and networked collaboration remain almost completely untapped. The forums are too crude, limited and don't scale well.

    I'm currently participating the Cathy Davidson's Coursera (Duke) "History and Future of (Mostly) Higher Education". Although she's trying to open up the format and encourage more horizontal collaboration, it's still very much an xMOOC (for edX, Udacity, Coursera etc), not a cMOOC (a reference to George Siemens' "Connectivist" approach to networked learning). The former aims to support the University (which is then able to serve students); the latter privileges the personal learning environment of the student over the strategic goals of the institution (although, for the moment, it assumes the institution is there, to provide the intellectual capital and publicly-funded labour). What is interesting about the #FutureEd MOOC is the strategy of encouraging nodal courses, discussion groups and related events outside Coursera's walled garden, using the coursers material as a set of resources and conversation starters. The dynamic between the inside and outside, this more open, ecological model, has potential to create additional value for both institutional and individual participants.

    ReplyDelete
  8. My friend and former Deloitte colleague just posted this blog post to his FB page. I wanted to share my comment (https://www.facebook.com/johnhagel/posts/10151903992747337?comment_id=33575914&offset=0&total_comments=2) here too as I think Mr. Cochrane makes some important points (by way of background, I'm the CTO at Relay Graduate School of Education in NYC, and we just released our first MOOC on Coursera). Here's what I wrote on John's page:

    "Mr. Cochrane's column talks about the difficulty in publishing and updating an on-line course. This is a real issue. It was some of my team who published (i.e., wrote the HTML for) for our Coursera course (https://www.coursera.org/course/teachingcharacter) - I think we pushed the limits of what can be done in Coursera pretty far (I think they agree too with that assertion). The problem of the 'concept to release' process for course publishing is something we're thinking a lot about at Relay, and in part why we continue to rely on our own LMS. I think one trick to wider adoption of MOOCs, and on-learning more generally, will be the 'portability' of courses for LMS to LMS. We're trying to design for that at Relay, thinking of the day soon when we'll want to publish courses to both our own LMS as well as other LMS systems, both public and private/proprietary. What we need is open standards and a single meta-language for on-line courses. There are some attempts out there but they have now been widely adopted."

    ReplyDelete
  9. This is one of the best piece of content on MOOC that i have read in recent times.

    ReplyDelete
  10. Great article. I would think that there will develop a market whereby municipalities and national governments fund part of MOOC platforms. These platforms are terrific for older less mobile adults as well as the poor. From a public policy standpoint, MOOCs are a public good. There will probably need to be done standardization. There will also need to be some sort of public badging or certificate involved.

    ReplyDelete
  11. Such a nice article on Mooconomics. I appreciate your work. Its one of the best article I have read on MOOC

    ReplyDelete
  12. This is perhaps one of the best pieces I have seen about "moocs" ... I actually heard about this on econtalk first and then landed here ... While the "social" aspects will remain important at universities - and while there is no real substitute for the in class teacher/mentor - good moocs will be able to replace average to poor teachers who are just "reading" the textbook and unable to provide context or real feedback to the students ...

    I am less worried/concerned about software and how crude it is today - While Star Trek may be a Hollywood creation, I can imagine a world where there is a "Commander Data" who can program fast and a computer that can understand the spoken word and replicate some of what a human teacher does (!)

    ReplyDelete

Comments are welcome. Keep it short, polite, and on topic.

Thanks to a few abusers I am now moderating comments. I welcome thoughtful disagreement. I will block comments with insulting or abusive language. I'm also blocking totally inane comments. Try to make some sense. I am much more likely to allow critical comments if you have the honesty and courage to use your real name.