C. Different stages in the development of the surveys

1. Explorative studies in pre-test phase

Before the first international study, there were a number of explorative studies starting with (1) gathering the existing information provided by other research, continuing with (2) several versions of provisional, non-scientific tests, and leading to (3) a scientific evaluation of the existing assessment tool, including suggestions for its further development.

1. Initial summary of existing church growth research (1986)

This phase was not focused on testing hypotheses, but on developing them by using a wide scope of empirical material. In a sabbatical that I took at Fuller Theological Seminary in Pasadena, California, in 1986, I studied all of the church growth books that were available by that time, and many academic papers that had not been published. Most of the material was based on case studies conducted in various parts of the world. I found many claims that the “principles” described were universal in nature (i.e., applying worldwide), but I could not find any proofs that would substantiate that claim.

On the basis of developing a “synopsis” of the different lists of features of growing churches (the terminology for referring to that reality differed immensely), I distinguished those features that the majority of authors had in common from those that were only shared by a few. This synopsis, combined with my own interviews with pastors of growing churches in the U.S. and beyond (some of them studying at Fuller), served as a first collection of ideas and potential hypotheses that could be utilized to develop an initial theoretical framework, on the basis of which empirical studies could be constructed.

2. Gathering further insights by non-scientific tests (1987–1993)

Based on these learnings that were accompanied by interviews with pastors of growing churches in Germany and Switzerland, I developed a first provisional test that I used with 120 churches in Germany, Switzerland, and Austria. While the procedures applied were not meant to be scientific at that stage, the first test urged me to develop an initial theory that could have the power of explaining church development. The results of those findings were published in the book, Praxis des Gemeindeaufbaus (1987).

In 1991, the procedure was further revised and extended to provide interested churches with a survey targeted at identifying their respective minimum factors, published in the book, Der Gemeindetest (1991). The test results we received through that offer, and the debriefing that I personally provided, enhanced my and my emerging team’s knowledge, leading to a revision of a number of the initially held hypotheses. In 1992 and 1993, I worked on a theological assessment of the empirical insights collected by that time, published in the book, Paradigmenwechsel in der Kirche (1993), which was later published in a revised form in English (Paradigm Shift in the Church, 1999).

Throughout this explorative phase, both the questionnaire and the overall theory were constantly adapted to new discoveries. One criterion applied from the outset was the distinction of growing and non-growing churches, a pre-scientific stage of developing the external criterion of correlating church quality with numerical growth.

3. Scientific evaluation of the existing assessment tool (1991–1993)

Overlapping with the previous phase, the first attempts directed at redeveloping the assessment procedure according to strict scientific standards began. This was primarily done by the organizational scientist Christoph Schalk, first in the course of presenting his evaluation of the procedures applied by that time in his dissertation, on which a large part of his paper Organizational Diagnosis of Churches is based. On the basis of the insights derived from that dissertation, Christoph developed suggestions of how the procedure could be revised in terms of classical test theory so that it satisfied recognized criteria like objectivity, reliability and validity. His suggestions resulted in a strongly revised form of questionnaire.

2. First international Study and repeat tests

After it had been field tested with churches in Germany, Austria, and Switzerland, the newly developed survey was used for studying the first sample of 1118 churches with 34,414 respondents in 32 countries. This is the research phase that is represented in the book, Natural Church Development.

Longitudinal studies)

This research resulted in the development of the NCD Church Profile in offer to churches since 1998, first in English and German, soon in a large number of other languages and countries. The evaluation of the data gathered this way and the debriefing that we provided led one more time to new discoveries and some revisions in the methodology.

In particular, the increasing number of churches taking retests provided insights that were not available to us before. The retests enabled us to monitor the development of churches, some of them over a period of more than twenty years, including the various ups and downs throughout that process.

Major revision of questionnaire

In 2005, we decided to implement a major revision of the questionnaire, primarily based on the learnings from people filling in the surveys in different countries and denominations. However, that revision had the disadvantage of making the comparison with previous profiles more difficult. Even if our learning continues constantly, apart from continually revising normations based on increasing data, we have to be extremely reluctant with major updates of that kind, since for the majority of churches the comparison with earlier results (and thus, the compatibility of the data) has become by far the most important piece of information provided by the survey.

Insights into change processes

The possibility of longitudinal studies on the development of churches over twenty years or more offers unique insights into actual change processes in individual churches and emerging trends in Christianity as a whole. Some of the insights gained from the change processes within these churches can be generalized to the majority of churches that are in comparable processes of change. Since NCD doesn’t feature a specific church model, but is strictly based on universal principles of church health and growth, even those churches experiencing health and growth that have never heard about “Natural Church Development” apply the same qualitative principles that NCD describes.

The most important lesson that we constantly learn is that change is indeed possible, and if it is pursued in a concerted effort by many church members—or even better, in a concerted effort by many churches within a given region—significant changes can be measured. Whenever we offered a two-year process to a group of interested churches in a given region, the average qualitative and quantitative scores at the end of those two years had been remarkably higher than they used to be at the beginning of the process. So far, there has not been any exception to this rule, and on the basis of our data it seems to be likely that such an exception may never emerge.

3. Secondary studies

The huge amount of data collected worldwide has made the NCD files an extremely promising base for conducting secondary studies, without the necessity of collecting new data by means of newly designed questionnaires. The secondary studies use the original raw data, including the possibility of measuring development over time.

Advantages and disadvantages of the NCD database

Using the data collected in the primary study of NCD together with the longitudinal studies that became possible recently, has both advantages and disadvantages.

● Among the advantages is the high number of churches represented (75,000), their international nature, the high number of repeat profiles, and the possibility of linking all observations to different levels of church quality. These are features that make the NCD database unique and highly promising for future research. This database enables us to detect even small indications of possible future trends.

● The disadvantage of using the NCD data for research projects are twofold. (1) As mentioned before, the data of the churches involved with an NCD process is not necessarily typical for “average” churches in a given country, denomination, or movement. However, since we can explore different quality levels and combine these observations, we may come close what the “average church” may be. (2) If we come up with new research topics (such as studying fundamentalism, the dynamics among “millennials,” or the correlation of liberal theology with both church health and growth), we must be reminded that the research methodology was originally not set up in order to answer these questions. We can only work with the questions that we have on our questionnaires, not with others that may be—in addition to them—worth asking.

However, in many cases the advantages mentioned outweigh the disadvantages. In the book, God Is Indestructible. Twelve Responses to the Relevance Crisis of Christianity I present a number of the results of the secondary studies mentioned.

One of the most important benefits of the NCD database should at least be mentioned in this context. Whenever I write on theological or observational topics, I check—primarily as a heuristic principle—the NCD database to find out if I can detect certain patterns connected to the questions at hand. This has influenced my writing even in the area of non-academic books, in which I don’t present all of the backgrounds of what led me to the statements I made. In my personal writing and teaching, everything goes through this filter, independent of the fact whether this heuristic process is documented (in academic books) or not (in the majority of the non-academic books). This has become a personal habit that is supposed to protect me from promoting my personal preferences and favorite ideas.

The difficulty of comparisons with non-NCD-shaped churches

Apart from the initial samples of churches in the explorative phase of the study, and the sample of 1,000 churches collected in the first international study, all of the data of churches in our files comes from churches that, with different degrees of commitment, are involved in an NCD process. Because of that, almost all of the churches in our files are “NCD-shaped churches.” This fact makes it difficult for us to compare these churches with those that have not been influenced by NCD. Some may view this as a purely “academic question,” as all our efforts are targeted at change processes that we have labeled Natural Church Development, but it is an important one.

The question that arises is, To what degree do the results that we come across apply to churches that have not been shaped by NCD?

● First, it has to be understood that the overwhelming majority of churches that initially take an NCD Survey have not been “shaped by NCD” before. In other words, the research results that we present reveal what can happen in churches regardless of which church model they may prioritize.

● Second, many denominations that use NCD consistently (for instance, at the level of a diocese or church district), like to compare some external results that are of particular importance to them—such as, giving habits, ministry to the poor, help for people who are HIV positive, participation in small groups, children’s ministry, reaching millennials, etc.—with reference to churches that are involved with an NCD process and those that are not. In every single case that I am aware of, the numbers of those involved with NCD are measurably higher, sometimes considerably. Nobody has to comment on these numbers, as they speak for themselves.

4. Detailed research through personal eTests

One of the possibilities for doing research that emerged more recently is the increasing number of eTests targeted at personal development. Of course, we have to do a lot of research as part of developing every single test. However, just as is the case with the NCD Church Survey, the automatically growing number of responses that we collect online enables us to get increasing insights into personal development processes that, in some cases, can even be more relevant than what we learn from the NCD Church Survey. Presently, these eTests are available in the areas of leadership (Empowerment Test), utilization of spiritual gifts (Gift Test), spirituality (Spiritual Style Test), life-supporting structures (Stewardship Test), essential dynamics of small groups (Energy Test), and a new paradigm for evangelism (Oikos Test).

Comparing the insights deriving from this data across different denominations, movements, and cultures has become one of the most promising paths of research for the future. The personal application of NCD principles is of utmost importance, as the decisive dynamics of church quality happen in the heads, hand, and hearts of the church members.

Since all of the tests are based on one of the quality characteristics of growing churches identified in the primary study of NCD, it is guaranteed that working with the eTests (particularly when not only used for diagnosis, but also for therapy) will contribute to the health and thus growth of a given church. The eTests supply an opportunity for every individual to contribute to the advance of Christianity, regardless of whether their own church leaders do or do not support these processes.

Scientific standards

It may be important to stress that, in providing the personal eTests, we are committed to the same scientific standards that are characteristic for the NCD Church Survey. In contrast to the Church Survey, the stress of the various eTests is on (a) reliability and (b) the normation supplied for various languages. As far as validity is concerned, in the realm of personal eTests, we apply the criterion of construct validity. Two of the most important procedures for the eTest development is factor analysis and internal consistency.

For all languages in which the various eTests are supplied, there is a normation based on the respective language. In contrast to the NCD Church Survey, however, we always start with an (albeit provisional) “international norm,” which enables us to supply immediate—even if less than accurate—feedback, and then proceed to language or country based norms depending on the size of data we collect. Whenever we reach a representative level of numbers of people researched, we apply the respective normations accordingly.

Qualitative research

In particular, in the area of the development of eTests, qualitative research (in addition to the quantitative research resulting, for instance, in different normations for different languages/countries) is applied. This qualitative research is not meant as a substitute for quantitative research, but aims at complementing it. In our case, qualitative research is primarily applied in the heuristic phase of test development, quantitative research, in the phase that is geared at offering the most exact results.

Much of the qualitative research that we conduct takes place through personal debriefing and coaching. It should be noted that all of these processes—and the respective learnings—take place at an international and thus, multicultural level. Since the overwhelming majority of this qualitative research takes place through video conferencing, i.e., Skype or Zoom meetings, it is possible to not only listen to the verbally articulated answers, but to monitor gestures, detect insecurities, and include non-verbal signals provided by the face (indicating interest or engagement) or the sound of the voice. In contrast to the standardized answer-scheme of a survey, that kind of qualitative research enables free and spontaneous answers to the questions.

By applying this procedure, we get invaluable insights into items that create difficulties in some cultural or denominational backgrounds. These insights help us revise the questions accordingly.

Evaluation of effectiveness

When we evaluated, in the area of eTests targeted at personal growth, all of the repeat test results, we could detect that in 79% of all cases, the coachees achieved higher scores in their repeat tests, compared to their initial results, i.e., they have visibly grown as a result of that process. Change is indeed possible, and in many contexts is not that difficult to achieve.