Automatic Testing of (RESTful) API Documentation

A presentation at APIdays Paris 2016 in December 2016 in Paris, France by Rouven Weßling

Slide 1

Slide 1

A ut omatic t esting o f (RES T f ul) API document ation A P I

Da y s

P a r i s ,

De c e m be r

2016 B y R ouv en Weßling ( ) Ec osyst em De v eloper / De v eloper Ev ang elist , Cont entf ul @R ouv enWessling p h ot o c r ed i t :

b y

M u s ée d u

L ou v r e D u s t i n G a ffk e (C C

B Y )

Slide 2

Slide 2

A c ont ent manag ement de v eloper platf orm with an API at its c or e.

Slide 3

Slide 3

Slide 4

Slide 4

What do y ou do ?

Slide 5

Slide 5

P o w er ed b y

Slide 6

Slide 6

Document ation will be wr ong

Slide 7

Slide 7

U ser s hat e that

Slide 8

Slide 8

I t ' s

a ll i n

t h e

s pe c

api blueprint A po w erf ul high-le v el API description languag e f or w eb APIs.

Slide 9

Slide 9

Q u e s t i o n s

C o l l e c t i o n

[ / q u e s t i o n s ]

C r e a t e

a

N e w

Q u e s t i o n

[ P O S T ]

Y o u

m a y

c r e a t e

y o u r

o w n

q u e s t i o n

u s i n g

t h i s

a c t i o n .

I t

t a k e s

a

J S O N

o b j e c t

c o n t a i n i n g

a

q u e s t i o n

a n d

a

c o l l e c t i o n

o f

a n s w e r s

i n

t h e

f o r m

o f

c h o i c e s .

R e q u e s t

( a p p l i c a t i o n / j s o n )

{

" q u e s t i o n " :

" F a v o u r i t e

p r o g r a m m i n g

l a n g u a g e ? " ,

" c h o i c e s " :

[

" S w i f t " ,

" P y t h o n " ,

" O b j e c t i v e

C " ,

" R u b y "

]

}

R e s p o n s e

2 0 1

( a p p l i c a t i o n / j s o n )

H e a d e r s

L o c a t i o n :

/ q u e s t i o n s / 2

B o d y

{

" q u e s t i o n " :

" F a v o u r i t e

p r o g r a m m i n g

l a n g u a g e ? " ,

" p u b l i s h e d _ a t " :

" 2 0 1 5

0 8

0 5 T 0 8 : 4 0 : 5 1 . 6 2 0 Z " ,

" c h o i c e s " :

[

{

" c h o i c e " :

" S w i f t " ,

Slide 10

Slide 10

L e t ' s

g e t

t e s t i n g DR E DD No mor e out dat ed API document ation.

Slide 11

Slide 11

T e s t i n g

r e a d- o n ly n o d e _ m o d u l e s / . b i n / d r e d d

c m a . a p i b

h t t p s : / / a p i . c o n t e n t f u l . c o m

\

m

G E T

Slide 12

Slide 12

Slide 13

Slide 13

Slide 14

Slide 14

H o o k s b e f o r e A l l

c a l l ed

a t

t h e b e g i n ni ng

of

t h e wh ol e t es t

ru n b e f o r e E a c h

c a l l ed

b ef or e e a c h

HT T P

t r a ns a c t i on b e f o r e

c a l l ed

b ef or e s om e s p ec i f i c

HT T P

t r a ns a c t i on b e f o r e E a c h V a l i d a t i o n

c a l l ed

b ef or e e a c h

HT T P

t r a ns a c t i on i s

v a l i d a t ed b e f o r e V a l i d a t i o n

c a l l ed

b ef or e s om e s p ec i f i c

HT T P

t r a ns a c t i on i s

v a l i d a t ed a f t e r

c a l l ed

a � er s om e s p ec i f i c

HT T P

t r a ns a c t i on r e g a r d l es s

i t s

r es u l t a f t e r E a c h

c a l l ed

a � er e a c h

HT T P

t r a ns a c t i on a f t e r A l l

c a l l ed

a � er wh ol e

t es t

ru n

Slide 15

Slide 15

H o o k s n o d e _ m o d u l e s / . b i n / d r e d d

c m a . a p i b

h t t p s : / / a p i . c o n t e n t f u l . c o m

\

h o o k f i l e s

. / t e s t

h o o k s . j s

\

m

G E T

Slide 16

Slide 16

S k i ppi n g

T e s t s v a r

h o o k s

=

r e q u i r e ( ' h o o k s ' ) ;

h o o k s . b e f o r e (

" W e b h o o k

c a l l s

W e b h o o k

c a l l

d e t a i l s

G e t

t h e

w e b h o o k

c a l l

d e t a i l s " ,

f u n c t i o n

( t r a n s a c t i o n )

{

t r a n s a c t i o n . s k i p

=

t r u e ;

} ) ;

Slide 17

Slide 17

Mu t a t i n g

d a t a v a r

h o o k s

=

r e q u i r e ( ' h o o k s ' ) ;

h o o k s . b e f o r e (

" E n t r i e s

D e l e t e

a n

E n t r y " ,

f u n c t i o n

( t r a n s a c t i o n )

{

c l i e n t . c r e a t e E n t r y ( e n t r y ,

" 1 2 3 4 " )

} ) ;

Slide 18

Slide 18

R a t e

li m i t i n g v a r

h o o k s

=

r e q u i r e ( ' h o o k s ' ) ;

c o n s t

D E L A Y

=

1 0 0 0 / 6 . 0 ;

h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,

d o n e )

{

s e t T i m e o u t ( d o n e ,

D E L A Y ) ;

} ) ;

Slide 19

Slide 19

C h a n g i n g

r e q u e s t

da t a v a r

h o o k s

=

r e q u i r e ( ' h o o k s ' ) ;

h o o k s . b e f o r e E a c h ( f u n c t i o n

( t r a n s a c t i o n ,

d o n e )

{

t r a n s a c t i o n . f u l l P a t h

=

t r a n s a c t i o n . f u l l P a t h . r e p l a c e ( ' f p 9 1 o e l s z i e a ' ,

' g b k x k l v m o l c 1 ' ) ;

d o n e ( ) ;

} ) ;

Slide 20

Slide 20

C h a i

a s s e r t i o n s v a r

h o o k s

=

r e q u i r e ( ' h o o k s ' ) ;

v a r

a s s e r t

=

r e q u i r e ( ' c h a i ' ) . a s s e r t ;

h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,

d o n e )

{

i f

( ! t r a n s a c t i o n . s k i p )

{

a s s e r t . m a t c h ( t r a n s a c t i o n . r e a l . h e a d e r s [ ' x

c o n t e n t f u l

r e q u e s t

i d ' ] ,

/ ^ c o n t e n t

a p i : [ a

z A

Z 0

9 ] { 2 2 } $ /

}

d o n e ( ) ;

} ) ;

Slide 21

Slide 21

S e c u r i t y

Slide 22

Slide 22

C e n s o r

pr i v a t e

da t a v a r

h o o k s

=

r e q u i r e ( ' h o o k s ' ) ;

c o n s t

S E C R E T _ H E A D E R S

=

[ ' A u t h o r i z a t i o n ' ] . m a p ( h e a d e r

=

h e a d e r . t o L o w e r C a s e ( ) ) ;

h o o k s . a f t e r E a c h V a l i d a t i o n ( f u n c t i o n ( t r a n s a c t i o n ,

d o n e )

{

O b j e c t . k e y s ( t r a n s a c t i o n . r e q u e s t . h e a d e r s ) . f o r E a c h ( f u n c t i o n ( k e y )

{

i f

( S E C R E T _ H E A D E R S . i n d e x O f ( k e y . t o L o w e r C a s e ( ) )

1 )

{

t r a n s a c t i o n . r e q u e s t . h e a d e r s [ k e y ]

=

" * * * H I D D E N

S E C R E T

D A T A * * * " ;

}

} ) ;

d o n e ( ) ;

} ) ;

Slide 23

Slide 23

C I

t h e

be a s t

Slide 24

Slide 24

Slide 25

Slide 25

C o n c lu s i o n Base y our document ation on an API spec T est that spec Mak e it p art of y our CI Don't substit ut e y our int e gr ation t ests

Slide 26

Slide 26

Slides av ailable on Slideshar e: http://www .slideshar e.ne t /r w essling / api-days

p aris -aut omatic -t esting-o f -r estf ul-api-document ation F ollo w me on T witt er: @R ouv enWessling