GAO Report on Higher Education: Use of New Data Could Help Improve Oversight of Distance Education

Main Findings

While distance education can use a variety of technologies, it has grown most rapidly online with the use of the Internet. Online distance education is currently being offered in various ways to students living on campus, away from a campus, and across state lines. School offerings in online learning range from individual classes to complete degree programs. Courses and degree programs may be a mix of face-to-face and online instruction – “hybrid” or “blended” instruction. Online asynchronous instruction—whereby students participate on their own schedule—is most common because it provides students with more convenience and flexibility, according to school officials. In the 2009-2010 academic year, almost half of postsecondary schools offered distance education opportunities to their students. Public 2- and 4-year schools were most likely to offer distance education, followed closely by private for-profit 4-year schools.

Students in distance education enroll mostly in public schools, and they represent a diverse population. While they tend to be older and female, and have family and work obligations, they also include students of all races, current and former members of the military, and those with disabilities. According to the most current Education data (2007-2008), students enrolled in distance education studied a range of subjects, such as business and health.

Accrediting agencies and schools assess the academic quality of distance education in several ways, but accreditors reported some oversight challenges. Federal law and regulations do not require accrediting agencies to have separate standards for reviewing distance education. As such, accreditors GAO spoke with have not adopted separate review standards, although they differed in the practices they used to examine schools offering distance education. Officials at two accreditors GAO spoke with cited some challenges with assessing quality, including keeping pace with the number of new online programs. School officials GAO interviewed reported using a range of design principles and student performance assessments to hold distance education to the same standards as face-to-face education. Some schools reported using specialized staff to translate face-to-face courses to the online environment, as well as standards developed by distance education experts to design their distance education courses. Schools also reported collecting outcome data, including data on student learning, to improve their courses.

Education has increased its monitoring of distance education but lacks sufficient data to inform its oversight activities. In 2009, Education began selecting 27 schools for distance education monitoring based on an analysis of risk factors, but it did not have data to identify schools with high enrollments in distance education, which may have impeded its ability to accurately identify high-risk schools. Between 2011 and 2013, Education’s National Center for Education Statistics (NCES) will start collecting survey data on the extent to which schools offer distance education, as well as enrollment levels. However, the department’s Office of Federal Student Aid (FSA), responsible for monitoring Title IV compliance, was not involved in the process of deciding what distance education information would be collected; therefore, it did not provide input on what types of data could be helpful in oversight. Further, FSA officials said they do not yet have a plan on how they will use the new data in monitoring.

Higher Education

As the largest provider of financial aid for postsecondary education, with about $134 billion provided to students during fiscal year 2010 under Title IV of the Higher Education Act of 1965, as amended (HEA),[1] the U.S. Department of Education (Education) has a considerable interest in distance education [2] – education that uses technology to provide instruction to students who are separated from the instructor. Students participating in distance education are eligible for federal financial aid in the same way as students taking traditional courses. This aid is available through Education in the form of grants, loans, and work study wages. While distance education has extended the opportunity to continue higher education learning and complete degrees to many students, including nontraditional students with work and family obligations as well as military personnel, its rapid growth in popularity presents challenges for ensuring both its quality and the oversight of federal funding. Congress has included a number of provisions in the HEA, as amended, such as oversight requirements for accrediting agencies, in recognition of the increasing role of distance education in higher education and to address concerns regarding the academic quality of distance education given the rapid growth in this area. Education’s Office of the Inspector General (OIG) has also raised concerns over the past few years about the ability of schools to verify student identities and ensure enrolled students are engaged in academic activities given the limited or no face-to-face contact with distance education students.

To provide a national perspective on the nature of distance education and its current level of oversight, we were asked to describe (1) the characteristics of distance education today, (2) the characteristics of students participating in distance education, (3) how the quality of distance education is being assessed, and (4) how Education monitors distance education in its stewardship of federal student aid funds.

To conduct our work, we reviewed and analyzed relevant federal laws and regulations, literature, studies, and program documents, and consulted with subject matter experts. We analyzed data from Education’s Integrated Postsecondary Education Data System (IPEDS) and the National Postsecondary Student Aid Study (NPSAS) databases to determine the school and student characteristics involved in distance education. We also reviewed information from a 2008 report by Education’s National Center for Education Statistics (NCES) and a 2010 industry report to obtain a national perspective on distance education practices and offerings at postsecondary schools. We conducted site visits to Florida, Minnesota, and Puerto Rico to visit schools and interview state agency officials. We selected these sites based on various factors, including the level of state data collected and an industry summary of states’ policies for approving distance education. We interviewed officials from higher education organizations, accrediting agencies (three regional, two national, and one specialized), and Education to determine their role in overseeing the delivery of distance education. Finally, we interviewed school officials from a nongeneralizable sample of 20 postsecondary schools to obtain more information on current practices in and the specific types of programs and coursework being offered through distance education. Our criteria for school selection included total enrollment, change in enrollment over time, school sector (public, private nonprofit, and private for-profit), and geographic diversity, among other factors. For more information on our scope and methodology, see appendix I.

We conducted this performance audit from November 2010 to November 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Background

Distance education is not a new concept, but in recent years, it has assumed markedly new forms and greater prominence. In the past, distance education generally took the form of correspondence courses— home study courses completed by mail. Distance education today can take many forms and is defined by federal law and regulation as education that uses one or more technologies (such as the Internet or audio conferencing) to deliver instruction to students who are separated from the instructor and to support regular and substantive interaction between the students and the instructor.[3] Instruction provided through the Internet—or online—may be synchronous (simultaneous or “real time”) or asynchronous, whereby students and the instructor need not be present and available at the same time (see Figure 1).

Figure 1: Illustration of Synchronous and Asynchronous Learning Methods.

Oversight

In general, for their students to be eligible for federal student aid funds under Title IV programs, schools must be legally authorized by a state, accredited by an agency recognized by Education, and be found eligible and certified by Education.[4] State governments, accrediting agencies, and Education form the program integrity triad established by Title IV of the HEA to oversee postsecondary education.[5] The state authorization role is primarily one of providing consumer protection through the state licensing process, while the accrediting agencies are intended to function as a quality assurance mechanism. In certifying a school for participation,[6] Education is responsible for determining the financial responsibility and administrative capability of schools and is also responsible for monitoring to ensure compliance with Title IV requirements.

Accrediting agencies, private educational associations set up to review the qualifications of member schools, are the primary overseers of schools’ academic quality. Accreditation is a peer review process that evaluates a school against the accrediting agency’s established standards. An institutional accrediting agency assesses a school in its entirety, including resources, admissions requirements, services offered, and the quality of its degree programs, while a programmatic accrediting agency reviews specific programs or single-purpose schools. A school’s accreditation is re-evaluated every 3 to 10 years, depending on the accrediting agency. If a school makes a substantive change to its educational programs or method of delivery from those that were offered when the agency last evaluated the school, the agency must ensure the change continues to meet standards.[7] Schools may lose accreditation if their accrediting agency determines that they no longer meet the established standards. While Education does not have the authority to dictate the specifics of an agency’s standards, the department recognizes accrediting agencies by reviewing and assessing their standards in various areas required by statute, such as student achievement, curricula, and student support services.[8]

Education’s Office of Federal Student Aid (FSA) is responsible for monitoring the over 6,000 postsecondary schools participating in Title IV programs to ensure their compliance with applicable statutory and regulatory provisions and to ensure that only eligible students receive federal student aid. The postsecondary school types include the following:

•       Public schools – schools operated and funded by state or local governments, including state universities and community colleges
•       Private nonprofit schools – schools owned and operated by nonprofit organizations whose net earnings do not benefit any shareholder or individual
•       For-profit schools – schools that are privately owned or owned by a publicly traded company and whose net earnings can benefit a shareholder or individual.

Education fulfills its school monitoring responsibilities through four main activities. First, it determines the initial eligibility of schools to participate in the federal student aid programs, as well as recertifies that eligibility periodically. Second, as part of ensuring compliance, FSA staff conduct program reviews of a select number of schools each year where they examine school records, interview school staff and students, and review relevant student information, among other things. FSA issues reports on these reviews, which include information on areas where a school was found to be in violation of the Title IV requirements. Third, schools are required to employ independent auditors to conduct annual compliance reviews and financial audits, which are then submitted to Education. Finally, Education’s OIG conducts its own audits and investigations of schools to identify and combat fraud, waste, and abuse and makes recommendations to the department. Education may assess liabilities and/or impose fines or other sanctions on schools found in violation of Title IV requirements.

Brief History of Statutory Provisions Related to Distance Education

Over the past two decades, Congress has made several changes to the Higher Education Act of 1965 that have affected schools’ offering, and accrediting agencies review, of distance education. To combat cases of fraud and abuse at postsecondary schools that primarily delivered distance education though correspondence courses, Congress stipulated in 1992 that schools were not eligible to participate in federal student aid programs if more than 50 percent of their courses were offered by correspondence or if more than 50 percent of their students were enrolled in correspondence courses. Furthermore, students enrolled in telecommunications courses were considered enrolled in a correspondence course if the sum of the telecommunications and correspondence courses equaled or exceeded 50 percent of the total courses offered at the school.[9] In 2006, Congress excluded telecommunication courses from the first two rules and eliminated the requirement that telecommunication courses be considered correspondence courses if the sum of both exceeded 50 percent of the total courses offered, allowing schools to expand distance education offerings while maintaining their eligibility to participate in Title IV programs.[10]^,[11]

While Congress has required that accrediting agencies apply and enforce standards with regard to distance education offered by schools, including that such reviews be included in their scope of accreditation since 1998,[12] it added additional requirements in 2008 through the Higher Education Opportunity Act (HEOA).[13] Although not required to have separate evaluation standards, accrediting agencies that have or are seeking to include distance education in their scope of review must demonstrate to Education that they effectively address the quality of a school’s distance education program in the same areas they are required to evaluate a school’s other educational offerings.[14] The agencies must require schools offering distance education to have processes to establish that the student who registers in a distance education course or program is the same student who participates in the program.[15] Additionally, agencies must ensure that accreditation team members whose responsibilities include evaluating distance education are well trained and knowledgeable with regard to distance education.[16] Finally, all accrediting agencies are required to monitor the growth of programs at institutions that are experiencing significant enrollment growth.[17]^,[18]

Distance Education Has Become Common in All Sectors and Is Offered througha Range of Programs and Courses

Online Distance Education Has Grown Dramatically and Is Offered in a Variety of Ways

While distance education can use various technologies, it has grown most rapidly online with the use of the Internet to support interaction among users. With the emergence of the Internet and expansion of Internet-based communication technologies, distance education today is a common phenomenon and widely used throughout higher education. Moreover, the term “distance education” no longer connotes only instruction separated by physical distance, since many distance education courses—specifically online courses—are offered to students living on campus as well as away from a campus and across state lines. School offerings in online learning range from individual classes to complete degree programs. Individual courses as well as degree programs may also be a mix of face-to-face and online instruction – often referred to as “hybrid” or “blended” instruction. Furthermore, an online class may be synchronous (simultaneous, real-time instruction), or asynchronous, where students and the instructor are not present and available at the same time.

According to a 2008 study on distance education conducted by Education, postsecondary schools of all types offer a variety of distance education courses.[19] Specifically, for the 2006-2007 school year, 61 percent of 2-year and 4-year schools reported offering online courses, 35 percent reported hybrid/blended courses, and 26 percent reported other types of distance courses. The study also suggests that the majority of schools offering distance education used asynchronous Internet technologies. Specifically, 92 percent of the degree-granting postsecondary institutions offering distance education in 2006-2007 reported using asynchronous Internet technologies to a moderate or large extent, compared with 31 percent of schools that reported using synchronous technologies to a moderate or large extent. In our interviews at the schools we selected, officials said that online, asynchronous instruction was also their predominant method for providing distance education and that this type of instruction meets students’ need for flexible schedules. For example, over half of the school officials we interviewed noted that many students taking classes online are working adults or active duty military service members who would otherwise be unable to continue or complete their studies.

The use of distance education, particularly online learning, has grown dramatically in recent years. According to a 2010 industry survey, online enrollment in degree-granting postsecondary schools has continued to grow at rates far in excess of the growth for total enrollment in higher education.[20] Survey results indicate that over 5.6 million students were taking at least one online course during the fall 2009 term – an increase of nearly 1 million students over the number reported the previous year and an increase of 21 percent, as compared with the less than 2 percent growth in the overall higher education student population. The survey also suggests that nearly 30 percent of higher education students were taking at least one course online.

Such remarkable growth may be attributed to institutional efforts to expand access to more students, alleviate constraints on campus capacity, and the desire to capitalize on emerging market opportunities and compete with other schools. According to Education’s 2008 study on distance education, which includes online and other forms of distance education, the top four factors affecting postsecondary schools’ decisions regarding distance education offerings are (1) meeting student demand for flexible schedules; (2) providing access to college for students who otherwise would not have access due to geographic, family, or work-related reasons; (3) making more courses available; and (4) seeking to increase student enrollment. Several of these factors, such as providing access to more students, were also cited by school officials we interviewed. For example, one school we visited had increased access to education by establishing over 20 “cyber-centers,” including one on a National Guard base and another in a shopping mall where students can access computers with Internet capabilities and participate in online courses as well as complete assignments and take exams. Additionally, officials at two of the schools we interviewed noted that on-campus students were registering for online classes, instead of face-to-face classes that were otherwise full or scheduled for times of day that conflicted with their personal schedules. Furthermore, one school we interviewed provided flexibility to its students by allowing them to begin and complete courses at their own pace.[21]

While cost savings might be a factor, none of the school officials we spoke with cited cost savings as the primary reason for providing online distance education courses and programs. Moreover, they said students taking distance education courses, including online courses, are generally charged the same tuition and fees as students taking face-to-face courses.[22] These officials cited various costs associated with developing and expanding online distance education offerings, such as the purchase of

Figure 2: Percentage of 2- and 4-Year Postsecondary Schools OfferingDistance Education, by Sector.

hardware and software (which includes a learning management system), course development, faculty training and salaries, and the provision of student support services. They also said online instruction is not necessarily less expensive to provide, in part, because schools have to provide similar support services to both online students and classroom students – such as tutoring, library access, and (virtual) faculty office hours. For example, officials at three schools mentioned one of the major expenses associated with online distance education is providing off-hours library access or tutoring. Also, almost all the officials said it is often difficult to isolate the costs of online courses from the costs of providing traditional courses. Professors generally teach both online and face-to-face course sections, and the infrastructure developed for online distance education, such as the online learning management systems, can also be used by students and instructors participating in face-to-face instruction.

A Wide Variety of Schools Provide a Range of Distance Education Courses and Programs

Schools of all types reported offering distance education, according to data collected by Education through its annual IPEDS survey.[23] Specifically, during the 2009-2010 school year, 46 percent of all Title IV eligible schools reported that they offered distance education opportunities to their students. Figure 2 shows the variation among these schools by sector and program length.

As shown in figure 2, public schools, both 2- and 4-year, were more likely to offer distance education opportunities than private nonprofit or for-profit schools. Among public schools, distance education was more likely to be offered at 2-year schools rather than 4-year schools. One school official we spoke with attributed this likelihood to the increased number of students at 2-year schools, given the weak economy and limited capacity at 4-year public schools.

With regard to minority-serving institutions and institutions with specific high minority concentrations,[24] IPEDS data indicate that these institutions are as likely or more likely to offer some distance education than all schools combined, with the exception of Hispanic-serving institutions. For the 2009-2010 school year, more than 60 percent of Historically Black Colleges and Universities and Tribal Colleges and Universities offered distance education opportunities to their students, compared with about 46 percent of institutions overall. Furthermore, 49 percent of Asian/Pacific Islander/Native Hawaiian-serving institutions offered distance education to their students. Among Hispanic-serving institutions, just over 30 percent of these schools were offering distance education (see fig. 3).[25]

Figure 3: Percentage of Minority-Serving Institutions Offering Distance Education.

With regard to the size of schools that offer some distance education, the IPEDS data suggest that larger schools—as defined by enrollment—are more likely to offer distance education opportunities than smaller schools. Specifically, 23 percent of schools with fewer than 1,000 students offered distance education, while 96 percent of larger schools—those with 20,000 or more students—did so (see fig. 4).

Figure 4:   Percentage of Postsecondary Schools Offering Distance Education, by School Size, 2009-2010

The 2008 distance education study by Education provided additional insights on the extent and nature of distance education offerings by school type, sector, and size. In terms of full degree and certificate programs, the study indicated that in the 2006-2007 academic year, about a third of all degree-granting schools offered entire degree programs or certificate programs through distance education. Additionally, public schools were more likely to offer a degree or certificate program entirely through distance education than were private schools. Larger schools were also more likely to offer a degree or certificate program entirely through distance education than smaller schools (see table 1).

Table 1:    Percentage of All Title IV Degree-Granting Postsecondary Schools 
                         Offering Degree or Certificate Programs Totally through Distance 
                         Education in Academic Year 2006-2007

Source: U.S. Department of Education.

Students in Distance Education Enroll Mostly in Public Schools and Represent a Diverse Population

Most Distance Education Students Attend Public Schools and Study a Range of Subjects

Our analysis of the NPSAS [26] data for the 2007-2008 academic year showed that of the estimated 5 million [27] postsecondary students who have taken distance education, participation was most common among students attending public schools. These students enrolled in a range of academic fields of study.

Most distance education students enroll at public schools. As might be expected, most undergraduate and graduate students taking distance education courses or programs were enrolled at public schools, followed by private nonprofit and private for-profit schools (see fig. 5).

Figure 5: Percentage of Students Enrolled in Distance Education, by Sector.

Figure 6:   Average Age of Undergraduate and Graduate Students, by Distance Education Enrollment Status.

Distance education students enroll in a variety of fields of study. Both undergraduate and graduate students taking distance education courses or programs had higher rates of enrollment in the fields of business and health. Undergraduates taking distance education courses and programs also often majored in the humanities (liberal arts), while graduate students often studied education.

While Students in Distance Education Tend to Be Older and Female, and Have Family and Work Obligations, They Are Also a Diverse Population

According to our analysis of 2007-2008 NPSAS data, distance education students varied somewhat from students who did not enroll in distance education in that they tended to be somewhat older and female, and have family and work obligations.[28]Moreover, students who are participating in distance education represent a diverse population that includes students of all races, current and former members of the military, and students with disabilities.[29] Some of these characteristics are consistent with what we reported in our 2002 testimony on distance education [30] and also were corroborated in our recent interviews with selected schools for this report.

Distance education students tend to be older. As figure 6 shows, undergraduate and graduate students who took distance education courses or programs were about 3 years older, on average, when compared with students who did not take any distance education courses.

Distance education students are more often female. Women represented about 61 percent of undergraduate students who took distance education courses or programs, compared with about 56 percent of undergraduates who took no distance education, and about 57 percent of undergraduates overall. For graduate students, the percentage of students taking distance education courses or programs who were female was about 65 percent, which was higher than those who took no distance education (59 percent) and the overall percentage of graduate students who were female (61 percent).

Distance education students more often have family obligations. Figure 7 shows that undergraduate and graduate students who took distance education courses or programs were more often married and had dependents than those taking no distance education courses.

Distance education students more often work full time. A higher percentage of students who took distance education courses or programs worked full time when compared with students who did not take any distance education courses. This difference was greatest among graduate students – about 74 percent of the students who took distance education courses or programs worked full time compared with 57 percent of students who did not take any distance education courses.[31] For undergraduates, the figures were 45 percent and 31 percent, respectively.[32]

Figure 7: Percentage of Undergraduate and Graduate Students with FamilyObligations, by Distance Education Enrollment Status.

Students of all races and ethnicities participate in distance education to some extent.[33] Postsecondary students of various races and ethnicities participated in distance education (see fig. 8).[34]

-

Figure 8:   Percentage of Students in Each Race and Ethnicity Group Who Participate in Distance Education.

Current and former members of the military enrolled in postsecondary education participate in distance education. Forty-five percent of active duty service members, 29 percent of reservists, and 30 percent of veterans enrolled in postsecondary education took distance education courses or programs.[35] In addition, of those enrolled in postsecondary education, 42 percent of active duty service members with a disability and 29 percent of veterans with a disability took distance education courses or programs.[36] Taken together, active duty service members, reservists, and veterans represented about 7 percent of all students taking distance education courses and programs, compared with 4 percent of students who took no distance education.

Students with disabilities participate in distance education. Twenty-one percent of all students with disabilities, including members of the military and civilians, enrolled in distance education courses or programs.[37] Further, 25 percent of students with disabilities affecting their mobility took distance education courses or programs. Students with disabilities represented 10 percent of all students taking distance education courses and programs, while students with mobility disabilities represented about 3 percent.[38]

Many of these student characteristics were also noted by school officials we interviewed. These school officials reported that they collect data such as age, gender, and race and ethnicity of their students. The demographic data provided from schools generally showed similar student characteristics as that suggested by the 2007-2008 NPSAS data – that distance education students tend to be older and female, and have work and family obligations. Officials of at least three of the schools we selected indicated that many of their students taking classes online are veterans or students serving in the military. While at least three schools reported tracking students who identified themselves as having disabilities, at the time of our interviews, none of these schools indicated that they had determined how many of these students were taking online distance education classes. Officials at one of these schools, however, conducted some analysis after our interview and reported that about 3 percent of their students enrolled in the past year had documented disabilities. These students took, on average, 15 percent of their classes online.[39]

While most of the schools where we conducted interviews collected demographic data on their students, including those taking courses online, less than half of these schools have compared the demographics of students taking completely online courses with those taking face-to-face courses. Officials at five schools mentioned that comparing data on students can be difficult, in part, because students can take courses or degrees through a mix of instructional modalities – including completely online, hybrid/blended (mix of online and face-to-face), and completely face-to-face. For example, officials from one private nonprofit 4-year school that offers completely online as well as blended courses and degrees said that it is difficult to collect comparison data because the school’s administrative records do not differentiate online students from those who enroll in both online and campus-based courses.

Accreditors and Schools Assess the Academic Quality of Distance Education in Several Ways, but Accreditors Reported Some Oversight Challenges

Accrediting Agencies Examine the Quality of Distance Education in Various Ways but Reported Some Challenges

Accreditors we interviewed have various procedures to examine schools’ distance education programs, but some accreditors reported they face challenges.[40] Federal law and regulations require accrediting agencies to have standards that address student achievement, curricula, faculty, and student support services, among other areas. In addition, accreditors must ensure that schools have a process in place to verify registered students are doing their own work by using methods such as secure logins, passwords, proctored examinations, or other technologies. However, accrediting agencies are not required to have separate standards for distance education.[41] As such, accreditors we spoke with who accredit both distance education and face-to-face programs use the same standards for both, although they differed in the practices they used to examine schools offering distance education.

The accreditors we spoke with conduct reviews of schools’ distance education programs according to the accreditors’ own standards. For example, to address the effectiveness of a program, accreditors may review such measures as student retention rates, completion/graduation rates, student satisfaction, placement rates (if applicable), and various measures of student learning. The three regional accreditors we spoke with give schools the responsibility for determining the best way to assess student learning for both face-to-face and distance education programs. However, both national accrediting agencies and the specialized accreditor we spoke with have specific quantitative thresholds as minimum standards on various outcomes. For example, one national accreditor requires that their member schools meet specific thresholds for student retention and placement rates. Officials at this agency said they could sanction schools whose programs fall below these standards. The other national accreditor we spoke with also requires its schools to meet thresholds established for outcomes such as course completion rates, program graduation rates, student satisfaction rates, and student learning (as measured by professional licensing exams such as those for physical therapists and lawyers). One regional accreditor said it was exploring including standardized learning outcomes in its accreditation standards.[42]

As part of their periodic site visits to schools to assess the quality of academic programs, accreditors have to adapt their approach when reviewing schools with distance education. Accreditors are required to employ staff who are well-trained and knowledgeable about distance education, for example, when performing on-site reviews of schools providing distance education.[43] Officials at all six accrediting agencies we spoke with said they include such experts on their on-site review teams. At one regional accreditor we interviewed, distance education experts are tasked with specifically reviewing the quality of a school’s distance education learning infrastructure, as well as the educational effectiveness of its programs, and receive specific training to do so. To review schools’ student supports, faculty supports, and educational effectiveness, officials at another regional accreditor told us their distance education experts may use video teleconferences or e-mails to communicate with administrative staff, faculty, and students not located on campus. These experts also remotely observe interactions between students and faculty in online classes.

In addition to the periodic on-site accreditation reviews to reassess a school’s accreditation status that are required by statute,[44] accreditors are to be notified if schools make substantive changes to academic programs or their schools.[45] The main purpose of this substantive change policy is to ensure that when schools make changes, they are maintaining the same level of quality they had when last reviewed. While there are a number of circumstances that can trigger the substantive change requirement, the one most applicable to distance education is the addition of courses or programs that represent a significant departure from the existing offerings of educational programs, including method of delivery, from those that were offered when the accreditor last evaluated the school. A shift to distance education courses that constitute more than 50 percent of a program’s offerings was the substantive change threshold used by four of the six accrediting agencies we interviewed. Officials at one regional accrediting agency reported that, in calendar year 2010, the agency turned down 34 percent of initial substantive change requests for new distance education programs because of weak student learning assessments or inadequately trained faculty, among other reasons. However, they said this figure has since come down to about 16 percent because schools have had more training on how to develop a substantive change proposal.

To ensure academic integrity, the six accrediting agencies we interviewed require schools to provide evidence that they verify registered students are doing their own work. For example, officials at one regional accreditor we spoke with said they require schools to use a student identification number and password as the minimum for verifying student identity. This accreditor said most institutions also verify student identity through student interaction during the course. In addition, one national accreditor we spoke with said some schools design tests that require a login and password, and may also feature pop-up questions during tests, prompting students to enter verification information such as their address or mother’s maiden name.

While the accreditors we interviewed have a range of activities to assess the quality of distance education, a few accrediting agency officials and industry experts we spoke with also expressed some concerns and reported challenges involved in assessing the quality of distance education. These challenges were mostly related to accreditors’ capacity to keep pace with substantive changes and conduct follow-up quality reviews with schools. Officials at one regional and one national accrediting agency said they have had some difficulty keeping up with the high number of substantive change applications for new online programs. Officials representing the national accreditor said these applications have increased by about 30 percent and that the officials have had to double the number of evaluators on staff over the last 5 years. According to officials with the regional accreditor, they have increased the number of follow-up reviews to ensure that schools address concerns about meeting quality standards identified during the initial site visits. These officials reported that they withdrew one school’s accreditation for failure to demonstrate that its distance education programs met the same standards as its face-to-face programs, with respect to curriculum, resources, support, and student learning outcomes. Industry experts also acknowledged that some accreditors have limited resources and have had problems training their peer reviewers in distance education.

Schools Use a Range of Course Design Principles and Student Performance Assessments to Hold Distance Education to the Same Quality Standards as Traditional Courses

To assure that their distance education programs are accredited by federally recognized accreditors and that their students qualify for Title IV funding, officials we interviewed at 20 selected schools reported that they generally apply certain course design principles and use student performance assessments to assess the quality of the courses that make up these programs. The accreditors we spoke with require schools to have standards that address the quality of degree programs with respect to such things as student achievement, which could include such measures as course completion, licensing exams, and job placement rates, as well as student support services. A majority of school officials reported that they assess their distance education courses by the same standards they use for their traditional courses.

Officials at most of the schools we spoke with said they used instructional teams to design their distance education courses according to the schools’ standards. These teams varied in their composition and activities.[46] Some teams include specialized staff who work with faculty to translate traditional face-to-face courses to the online environment. For example, one school we visited in Florida has a 20-member instructional design team that includes instructional designers, graphic artists, multimedia technicians, and quality control coordinators. Officials at this school said the design team considers which instructional methods are most appropriate for the material delivered in each online course. For example, a psychology course may use mostly text-based storytelling, while an anthropology class may rely more heavily on video clips. Officials at an online school we spoke with stressed the need to replace face-to-face course instructors’ body language and tone of voice cues with appropriate text and video media. Besides assisting professors with designing online courses, school officials said instructional design teams also train professors in the pedagogical differences of teaching online and on the online technology used by the school.

Officials at over half of the 20 schools we interviewed also reported that, to ensure quality in the design of their courses, they had used standards and best practices, some of which were developed by distance education industry experts. For example, 5 schools subscribe to Quality Matters, a nonprofit organization that lays out principles for designing quality online and blended courses. This organization sets specific standards for learning objectives, technology, faculty-student interaction, student supports, and assessment that online courses must meet in order to receive Quality Matters certification.

In addition, school officials reported that their schools collect outcome data to help them assess the quality of courses. The types of learning outcomes that the schools reported tracking include end-of-course grades, course completion rates, and results of national professional licensing assessments. Officials at most schools we spoke with said they also used outcome data to make improvements to their courses. Officials at two schools told us they employ staff to analyze these data and make recommendations for course updates. For example, officials at one fully online school we spoke with noticed their students were performing below the national average on a section of a third-party end-of-course criminal justice test. The officials used the results of this test to strengthen the related material. According to these officials, their criminal justice students’ performance improved on that section of the exam subsequent to their course improvements. In addition to using outcome data to improve their courses, one school we spoke with in Florida had collected these types of data on their online and hybrid courses over a period of 15 years to determine which factors most influenced student success.

To meet accreditors’ requirement to verify the identities of students enrolled in their distance education courses or programs, officials at most of the schools reported using various methods. For example, most of the school officials we interviewed said they issue students a secure login and password and some also use other methods, such as proctored exams. Officials at one school said they are also starting to use audiovisual software that works as a web cam to verify the student taking an exam is the one enrolled in the course and to ensure the student is not receiving assistance. In addition to technological safeguards, officials from one school said the interaction between students and faculty is key to ensuring students are doing their own work. They said instructors become familiar with a student’s writing or communication style through online discussions or the completion of assignments, and the instructor recognizes if that style changes. Officials at one school said they cannot be completely sure that distance education students are doing all of their own work even when using these methods; officials also noted that similar challenges exist for face-to-face courses. A few schools mentioned taking further steps to combat potential fraud in their online programs. Specifically, officials at two of the completely online schools we interviewed said they conduct reviews of or request further documentation from students who register with the same e-mail addresses or telephone numbers.[47] Officials at one school we spoke with said they would like more guidance, either from Education or their institutional accrediting agency, on examples of verification and authentication systems for student identity to improve the school’s monitoring of the verification process.

Education Has Increased Its Monitoring of Distance Education but Lacks Sufficient Data to Inform Its Oversight

Education’s Office of Federal Student Aid (FSA) has recently increased its monitoring of distance education by updating its program review procedures and undertaking a risk analysis project. These efforts are in response to the expansion of distance education and the Education OIG’s identification of distance education as a high-risk area for managing student aid dollars.[48] To better monitor distance education, FSA updated and issued new program review procedures. The previous set of FSA’s procedures, issued in 2008, did not provide in-depth guidance for assessing whether a school was approved to offer distance education or if there was regular and substantive interaction between instructors and students. The new procedures on distance education provide staff with expanded guidance for assessing a school’s compliance with these requirements. FSA officials said staff have been trained on the new procedures and, as of June 2011, have been using them for program reviews.[49] All program reviews will include at least routine testing to determine basic program eligibility for schools that offer distance education, according to Education officials. Schools that offer more than half of any of their programs through distance education will also be required to undergo expanded testing for regular and substantive interaction.

Compliance with federal student aid requirements by schools offering distance education programs is difficult to assess because many of the violations Education identifies through its program reviews are not specific to distance education; for those that are, Education does not necessarily identify or code the violations as such in its database, according to an Education official. For example, violations such as a school not appropriately returning Title IV funds when a student withdraws are coded in Education’s database based on the type of violation rather than whether this violation occurred in traditional or distance education. Violations specific to distance education that are tracked by Education are related to a lack of regular and substantive interaction between instructor and students and certain accreditation issues, such as an accrediting agency being ineligible because it does not have distance education in its scope. Education reported that from October 2005 through May 2011, no program reviews or audits identified any lack of regular and substantive interaction or distance education accreditation violations.[50]

In addition to FSA updating the program review procedures, in 2009, OIG and FSA, at the request of the Deputy Undersecretary, initiated a project analyzing risk factors for noncompliance with Title IV requirements by schools offering distance education. Under this project, 27 schools were selected for review based on a set of indicators OIG and FSA considered to be high risk for noncompliance, such as a large change in the amount of federal student aid a school is receiving.[51]^,[52] The OIG/FSA group is conducting this project in conjunction with others in the department.[53] FSA officials were not able to estimate a date when all final project reports will be issued, but said their last program review was conducted in early August 2011. They said the results of the project, including its methods for identifying high-risk schools and the procedures used, will be evaluated to determine if any changes need to be made to FSA’s annual program reviews.

While the objective of the project was to review high-risk distance education schools, Education lacked data to adequately identify schools’ level of risk based on the extent to which they offered distance education and the amount of federal student aid they received for those programs or courses. For example, to identify high-risk schools that may be offering distance education courses and programs, one indicator Education relied on was the Department of Defense’s enrollment information on its military members. Because distance education provides the flexibility needed to fit active duty service members’ duty schedules and location, many military members are enrolled in distance education courses and programs.[54]Therefore, in its risk analysis, Education included schools that had 200 or more military members receiving tuition assistance from the Department of Defense. While Education’s IPEDS database can show which schools offer distance education, it lacks information on the extent of a school’s offerings and enrollment levels. Despite using data from multiple sources, one of the 27 schools Education originally selected for review through the risk analysis did not actually offer distance education. As a result, FSA officials said they had to substitute another school for the study. While the project is not yet complete, officials reported confidence that their study is currently based on an appropriate selection of schools.[55] Nevertheless, they acknowledged that, in selecting their target schools, they lacked sufficient data to help them identify the extent to which a school was offering distance education as well as the amount of federal dollars being spent for distance education at each school, both of which would have been significant in evaluating a school’s risk.

The Office of Federal Student Aid has plans to collect more information on distance education, but complete information on all schools may not be available for several years. Under its new Integrated Partner Management (IPM) system, which will consolidate data systems on schools receiving Title IV funds, FSA will collect information about how a school’s programs are offered. Specifically, FSA officials said when schools apply for Title IV initial certification or recertification, they will be asked to indicate whether a program is predominantly (more than 50 percent) delivered via the classroom, distance education, correspondence, or independent study. They said the IPM system is expected to be implemented in November 2012 and would eventually allow them to analyze comprehensive data about a school. For example, they will be able to match the extent to which schools offer distance education with Title IV violations identified during program reviews. However, because schools are generally required to recertify only every 6 years, officials acknowledged that it could be several years before the IPM system will contain information on all schools’ distance education offerings.[56] Therefore, distance education information on all schools may not be available through IPM until 2018.

In the meantime, Education’s NCES is expanding its IPEDS survey to provide a more in-depth picture of distance education offerings and enrollment patterns.[57] The plan by NCES to expand the IPEDS survey with regard to distance education was the result of a decision by its technical review panel to better describe postsecondary education offered throughout the nation, allow schools to compare their distance education activities with those of their peer schools, and provide valuable information to parents and students on available college programs. This expanded data collection will be conducted in phases. The 2011-2012 survey used the definition of distance education as established in 2008 and collected information about whether schools offer their programs completely through distance education. Additional new distance education questions will be added to the 2012-2013 survey. The new survey questions ask for information such as the range of a school’s offerings in distance education, the number of students enrolled either partially or entirely in distance education, and whether the students are located in or out of state in relation to the school (see fig. 9). An NCES official said the new IPEDS data are expected to be available 1 year after the survey closes but may be available earlier. For example, early release data collected during the

^a The classification instructional program code is a coding scheme that contains titles and descriptions of instructional programs.

Figure 9: IPEDS Changes Related to Distance Education.

2011-2012 survey may be available as early as February 2012 and available publicly by November 2012.

Despite the prospect of more comprehensive data on schools and their distance education offerings being collected through IPEDS, FSA does not yet have specific plans to use these data for monitoring school compliance with federal student aid requirements. According to FSA officials, they intend to wait and see what information the survey yields before deciding how to make use of it.[58] Moreover, FSA indicated it was not aware of NCES’s efforts to expand the IPEDS distance education data collection and, therefore, was not involved in the planning and did not provide input during comment periods.[59] According to NCES officials, the NCES technical review panel process engages a number of stakeholders and is open to federal officials who are interested in participating.

Conclusions

Distance education, specifically online education, has been developing for a number of years and has become a part of the mainstream of higher education. This delivery mode of instruction has provided some new opportunities and access, particularly for nontraditional students and working adults who are looking to advance their careers. Moreover, it is likely to continue growing, as schools across all sectors and levels see it as a critical educational tool in meeting student needs and demand.

The growth in distance education and the sizable federal investment in higher education will challenge all segments of the triad responsible for the oversight of higher education—the states, accreditation agencies, and the federal government—in their capacity to provide consumer protection, ensure academic quality, and protect the federal investment. In response to this challenge, Education has taken steps to increase its oversight by providing its staff with expanded guidance for assessing a school’s compliance with distance education requirements and participating in the OIG/FSA risk project, which identified potential risk indicators. However, a key factor in Education’s ability to properly focus oversight on the areas of greatest risk will be the availability and use of pertinent, up-to-date data on both the extent to which schools offer distance education and the extent to which students use federal aid to attend those programs.

While FSA’s IPM system may eventually be helpful in providing Education with the opportunity to monitor distance education with better information, the expanded IPEDS data would provide relevant information much sooner. However, without a plan on how to use the new IPEDS data to identify and monitor high-risk schools, FSA may lose the opportunity to strengthen its oversight of distance education in the near term. Moreover, if FSA does not coordinate with NCES going forward, it stands to lose the opportunity to provide input on any additionally needed data that may strengthen oversight and ensure accountability in the long term.

Recommendations for Executive Action

To help Education strengthen its oversight of distance education, the Secretary of Education should direct FSA to develop a plan on how best to use the new IPEDS distance education data and provide input to NCES on future IPEDS survey work with regard to distance education.

Agency Comments and Our Evaluation

We provided a draft of this report to officials at Education for their review and comment. Education provided comments, which are reproduced in appendix III of the full text report, and technical comments, which we incorporated as appropriate. In its comments, Education agreed with our recommendation and noted that FSA will update its School Participation Team procedures to include consideration of IPEDS data on distance education for monitoring schools. Education also stated that FSA will provide input to NCES on the design and results of any future IPEDS surveys that include distance education.

We are sending copies of this report to relevant congressional committees, the Secretary of Education, and other interested parties. In addition, this report will also be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

George A. Scott

Director, Education, Workforce, and Income Security Issues

Appendix I: Objectives, Scope, and Methodology

This appendix discusses in detail our methodology for addressing the following research objectives: (1) the characteristics of distance education today, (2) the characteristics of students participating in distance education, (3) how the quality of distance education is being assessed, and (4) how Education monitors distance education in its stewardship of federal student aid funds.

To address these research questions, we reviewed relevant federal laws and regulations, literature, studies, and reports; interviewed officials from Education, representatives from all types of postsecondary schools, accreditation agencies, and distance education and industry experts; and conducted site visits to Florida, Minnesota, and Puerto Rico to interview state agency and school officials. We selected these sites based on various factors, including the level of state data collected and an industry summary of states’ policies for approving distance education. We also analyzed data from Education’s Integrated Postsecondary Education Data System (IPEDS) and the National Postsecondary Student Aid Study (NPSAS) databases to determine the school and student characteristics involved in distance education. We determined that IPEDS and NPSAS data were sufficiently reliable for the purposes of this report based on prior testing of the data from these systems in 2011.[60] The data were tested for accuracy and completeness, documentation about the data and systems used to produce the data was reviewed, and agency officials were interviewed.

To determine the current characteristics of distance education, we analyzed 2009-2010 data from Education’s IPEDS and also from a 2008 report by Education’s National Center for Education Statistics (NCES) [61] to obtain a national perspective on distance education practices and offerings at postsecondary schools. Specifically, we analyzed IPEDS data to provide information on the size, number, sector, and program length of schools offering distance education courses and programs. We used the 2008 distance education report to describe how schools are providing distance education to students, including the type of technology (Internet, video, audio, etc.) and instructional methods (asynchronous and synchronous) used, and the various types of degrees, certificates, and courses offered, including the percentage of courses offered online. In addition, we analyzed the 2010 Sloan Consortium [62] report on online education to show updated enrollment figures specific to online courses.[63] We supplemented the nationally representative data with information obtained from our interviews with industry experts and representatives at a nongeneralizable sample of postsecondary schools regarding the range of delivery and instructional techniques being used, and the type of programs and coursework offered through distance education.

To select our sample of postsecondary schools, we used enrollment data from Education’s 2009-2010 IPEDS to identify schools that were offering distance education and had significant increases in total enrollment, which may be due, in part, to increased enrollment in distance education classes or programs. Based on the schools’ percentage change in enrollment, we then selected schools by size—as defined by enrollment—as well as by sector and program length. We also considered the following factors in selecting our sample of schools:

•       geographic dispersion by state
•       minority serving school status (e.g., Historically Black Colleges and Universities and Hispanic-serving institutions)
•       selectivity in accepting students
•       industry expert or stakeholder recommendations
•       extent to which distance education programs and courses are offered (totally online schools versus schools offering both campus-based and online instruction), and
•       whether the schools are regionally or nationally accredited.

Based on these considerations, we selected 20 schools representing all sectors and program lengths, for site visits or phone interviews (see app. II for a list of colleges and universities we interviewed). Our selected schools break out as follows:

•       4 public 2-year schools
•       5 public 4-year schools
•       6 private nonprofit schools
•       5 private for-profit schools        

After our interviews with officials from the selected schools, we conducted a content analysis on the information gathered. Interview responses and comments from officials were categorized to identify common themes. The themes were reviewed by a methodologist before all comments were categorized. One analyst coded the information and a second analyst assessed the accuracy of the coding. Disagreements between coders were resolved through discussion. We used the information gathered from these schools for illustrative purposes only. Because the schools were not selected to be representative of all postsecondary schools, the interview results are not generalizable to other postsecondary schools, including groups of schools in the same sector or program length.

To determine the characteristics of students participating in distance education courses and programs, as well as those who do not participate, we analyzed Education’s 2007-2008 NPSAS data, the most current available data.[64] These data allowed us to compare distance education students to nondistance education students on the following characteristics: age, gender, marital status, dependent status, and employment status. The data also allowed us to describe the characteristics of students enrolled in distance education, in terms of type of school attended, field of study, race, veteran status, and disability status. We supplemented this analysis with information from our interviews with selected postsecondary schools and student demographic data provided by school officials.

To determine how the quality of distance education programs is being assessed, we obtained information from accrediting agency and school officials and reviewed and analyzed federal laws and regulations related to accreditation. We interviewed officials from six accrediting agencies (three regional, two national, and one specialized [65]) and reviewed their standards and policies to determine how they are assessing the quality of distance education courses and programs. In addition, we reviewed documents from the Council for Higher Education Accreditation (CHEA) [66] website, to gain a broader understanding of accreditation. We also interviewed officials from schools in our sample to describe the specific quality assurance frameworks and the outcomes they use to assess the performance of students engaged in distance education. In addition, we interviewed an official from Quality Matters and reviewed quality standards documents provided at the interview.[67]

To determine the extent to which Education is monitoring distance education programs to ensure the protection of federal student aid funds, we reviewed relevant federal laws and regulations regarding distance education oversight requirements. We interviewed officials from Education’s Federal Student Aid office and the Office of Postsecondary Education to determine their roles in the monitoring and governance of Title IV programs, specifically with respect to distance education. In addition, we interviewed officials from NCES to learn about their IPEDS data collection efforts and Education’s Office of the Inspector General to learn about their distance education monitoring activities and findings. Finally, we reviewed agency documents, including plans to add distance education variables to the IPEDS survey, OIG testimonies and reports, and an interim status memorandum issued by the OIG/FSA Risk Project.

We conducted this performance audit from November 2010 to November 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: List of Colleges and Universities GAO Interviewed

   Source: GAO.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

George A. Scott, (202) 512-7215 or scottg@gao.gov

Staff Acknowledgments

In addition to the contact named above, Tranchau Nguyen, Assistant Director; Susan Chin, Analyst-in-Charge; Amy Anderson; Jeffrey G. Miller; and Jodi Munson Rodríguez made significant contributions to this report in all aspects of the work. Susan Bernstein contributed to writing this report. Michael Silver, Christine San, and John Mingus provided technical support, and Jessica Botsford provided legal support. Mimi Nguyen assisted with report graphics.

Related GAO Products

Distance Education: Growth in Distance Education Programs and Implications for Federal Education Policy GAO-02-1125T (Washington, D.C.: GAO, 26 September 2002); available at www.gao.gov/products/GAO-02-1125T.

Distance Education: Improved Data on Program Costs and Guidelines on Quality Assessments Needed to Inform Federal Policy, GAO-04-279 (Washington, D.C.: GAO, 26 February 2004); available at www.gao.gov/products/GAO-04-279.

Higher Education: Institutions’ Reported Data Collection Burden is Higher Than Estimated but Can Be Reduced through Increased Coordination, GAO-10-871 (Washington, D.C.: GAO, 13 August 2010); available at www.gao.gov/products/GAO-10-871.

DOD Education Benefits: Increased Oversight of Tuition Assistance Program Is Needed, GAO-11-300 (Washington, D.C.: GAO, 1 March 2011); available at www.gao.gov/products/GAO-11-300.

DOD Education Benefits: Further Actions Needed to Improve Oversight of Tuition Assistance Program, GAO-11-389T (Washington, D.C.: GAO, 2 March 2011); available at www.gao.gov/products/GAO-11-389T.

VA Education Benefits: Actions Taken, but Outreach and Oversight Could Be Improved, GAO-11-256 (Washington, D.C.: GAO, 28 February 2011); available at www.gao.gov/products/GAO-11-256.