Automatic Testing of (RESTful) API Documentation
A presentation at APIdays Paris 2016 in in Paris, France by Rouven Weßling
A ut omatic t esting o f (RES T f ul) API document ation A P I
Da y s
P a r i s ,
De c e m be r
2016 B y R ouv en Weßling ( ) Ec osyst em De v eloper / De v eloper Ev ang elist , Cont entf ul @R ouv enWessling p h ot o c r ed i t :
b y
M u s ée d u
L ou v r e D u s t i n G a ffk e (C C
B Y )
A c ont ent manag ement de v eloper platf orm with an API at its c or e.
What do y ou do ?
P o w er ed b y
Document ation will be wr ong
U ser s hat e that
I t ' s
a ll i n
t h e
s pe c
api blueprint A po w erf ul high-le v el API description languag e f or w eb APIs.
Q u e s t i o n s
C o l l e c t i o n
[ / q u e s t i o n s ]
C r e a t e
a
N e w
Q u e s t i o n
[ P O S T ]
Y o u
m a y
c r e a t e
y o u r
o w n
q u e s t i o n
u s i n g
t h i s
a c t i o n .
I t
t a k e s
a
J S O N
o b j e c t
c o n t a i n i n g
a
q u e s t i o n
a n d
a
c o l l e c t i o n
o f
a n s w e r s
i n
t h e
f o r m
o f
c h o i c e s .
R e q u e s t
( a p p l i c a t i o n / j s o n )
{
" q u e s t i o n " :
" F a v o u r i t e
p r o g r a m m i n g
l a n g u a g e ? " ,
" c h o i c e s " :
[
" S w i f t " ,
" P y t h o n " ,
" O b j e c t i v e
C " ,
" R u b y "
]
}
R e s p o n s e
2 0 1
( a p p l i c a t i o n / j s o n )
H e a d e r s
L o c a t i o n :
/ q u e s t i o n s / 2
B o d y
{
" q u e s t i o n " :
" F a v o u r i t e
p r o g r a m m i n g
l a n g u a g e ? " ,
" p u b l i s h e d _ a t " :
" 2 0 1 5
0 8
0 5 T 0 8 : 4 0 : 5 1 . 6 2 0 Z " ,
" c h o i c e s " :
[
{
" c h o i c e " :
" S w i f t " ,
L e t ' s
g e t
t e s t i n g DR E DD No mor e out dat ed API document ation.
T e s t i n g
r e a d- o n ly n o d e _ m o d u l e s / . b i n / d r e d d
c m a . a p i b
h t t p s : / / a p i . c o n t e n t f u l . c o m
\
m
G E T
H o o k s b e f o r e A l l
c a l l ed
a t
t h e b e g i n ni ng
of
t h e wh ol e t es t
ru n b e f o r e E a c h
c a l l ed
b ef or e e a c h
HT T P
t r a ns a c t i on b e f o r e
c a l l ed
b ef or e s om e s p ec i f i c
HT T P
t r a ns a c t i on b e f o r e E a c h V a l i d a t i o n
c a l l ed
b ef or e e a c h
HT T P
t r a ns a c t i on i s
v a l i d a t ed b e f o r e V a l i d a t i o n
c a l l ed
b ef or e s om e s p ec i f i c
HT T P
t r a ns a c t i on i s
v a l i d a t ed a f t e r
c a l l ed
a � er s om e s p ec i f i c
HT T P
t r a ns a c t i on r e g a r d l es s
i t s
r es u l t a f t e r E a c h
c a l l ed
a � er e a c h
HT T P
t r a ns a c t i on a f t e r A l l
c a l l ed
a � er wh ol e
t es t
ru n
H o o k s n o d e _ m o d u l e s / . b i n / d r e d d
c m a . a p i b
h t t p s : / / a p i . c o n t e n t f u l . c o m
\
h o o k f i l e s
. / t e s t
h o o k s . j s
\
m
G E T
S k i ppi n g
T e s t s v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ;
h o o k s . b e f o r e (
" W e b h o o k
c a l l s
W e b h o o k
c a l l
d e t a i l s
G e t
t h e
w e b h o o k
c a l l
d e t a i l s " ,
f u n c t i o n
( t r a n s a c t i o n )
{
t r a n s a c t i o n . s k i p
=
t r u e ;
} ) ;
Mu t a t i n g
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ;
h o o k s . b e f o r e (
" E n t r i e s
D e l e t e
a n
E n t r y " ,
f u n c t i o n
( t r a n s a c t i o n )
{
c l i e n t . c r e a t e E n t r y ( e n t r y ,
" 1 2 3 4 " )
} ) ;
R a t e
li m i t i n g v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ;
c o n s t
D E L A Y
=
1 0 0 0 / 6 . 0 ;
h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
s e t T i m e o u t ( d o n e ,
D E L A Y ) ;
} ) ;
C h a n g i n g
r e q u e s t
da t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ;
h o o k s . b e f o r e E a c h ( f u n c t i o n
( t r a n s a c t i o n ,
d o n e )
{
t r a n s a c t i o n . f u l l P a t h
=
t r a n s a c t i o n . f u l l P a t h . r e p l a c e ( ' f p 9 1 o e l s z i e a ' ,
' g b k x k l v m o l c 1 ' ) ;
d o n e ( ) ;
} ) ;
C h a i
a s s e r t i o n s v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ;
v a r
a s s e r t
=
r e q u i r e ( ' c h a i ' ) . a s s e r t ;
h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
i f
( ! t r a n s a c t i o n . s k i p )
{
a s s e r t . m a t c h ( t r a n s a c t i o n . r e a l . h e a d e r s [ ' x
c o n t e n t f u l
r e q u e s t
i d ' ] ,
/ ^ c o n t e n t
a p i : [ a
z A
Z 0
9 ] { 2 2 } $ /
}
d o n e ( ) ;
} ) ;
S e c u r i t y
�
C e n s o r
pr i v a t e
da t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ;
c o n s t
S E C R E T _ H E A D E R S
=
[ ' A u t h o r i z a t i o n ' ] . m a p ( h e a d e r
=
h e a d e r . t o L o w e r C a s e ( ) ) ;
h o o k s . a f t e r E a c h V a l i d a t i o n ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
O b j e c t . k e y s ( t r a n s a c t i o n . r e q u e s t . h e a d e r s ) . f o r E a c h ( f u n c t i o n ( k e y )
{
i f
( S E C R E T _ H E A D E R S . i n d e x O f ( k e y . t o L o w e r C a s e ( ) )
1 )
{
t r a n s a c t i o n . r e q u e s t . h e a d e r s [ k e y ]
=
" * * * H I D D E N
S E C R E T
D A T A * * * " ;
}
} ) ;
d o n e ( ) ;
} ) ;
C I
t h e
be a s t
C o n c lu s i o n Base y our document ation on an API spec T est that spec Mak e it p art of y our CI Don't substit ut e y our int e gr ation t ests
Slides av ailable on Slideshar e: http://www .slideshar e.ne t /r w essling / api-days
p aris -aut omatic -t esting-o f -r estf ul-api-document ation F ollo w me on T witt er: @R ouv enWessling
There are few things more frustrating than documentation that doesn’t match the implementation – especially when it’s documentation for your customers. In actively developed software that is a reoccurring problem. As the product evolves the documentation often lags behind and sometimes breaking changes sneak it, breaking the very examples that are part of the documentation. To avoid all these problems, it’s time to start testing your docs. In this talk I’ll show you what’s currently possible when it comes to testing documentation, how to start testing API documentation written using API Blueprints and how to incorporate it into your workflow.
Buzz and feedback
Here’s what was said about this presentation on social media.
