A presentation at Nordic APIs Platform Summit in in Stockholm, Sweden by Rouven Weßling
A u t o m a t i c
t e s t i n g
o f
( R E S T f u l ) A P I
d o c u m e n t a t i o n N o r d i c
A P I s
P l a t f o r m
S u m m i t ,
O c t o b e r
2 0 1 6 B y
R o u v e n
W e ß l i n g
( ) E c o s y s t e m
D e v e l o p e r
/
D e v e l o p e r
E v a n g e l i s t ,
C o n t e n t f u l @ R o u v e n W e s s l i n g p h o t o
c r e d i t :
b y
S t o c k h o l m ,
A p p r o a c h i n g
B l u e
H o u r T o b i a s
B Y )
A
c o n t e n t
m a n a g e m e n t
d e v e l o p e r
p l a t f o r m
w i t h
a n
A P I
a t
i t s
c o r e .
W h a t
d o
y o u
d o ?
P o w e r e d
b y
D o c u m e n t a t i o n
w i l l
b e
w r o n g
U s e r s
h a t e
t h a t
I t ' s
a l l
i n
t h e
s p e c
a p i
b l u e p r i n t A
p o w e r f u l
l e v e l
A P I
d e s c r i p t i o n
l a n g u a g e
f o r
w e b
A P I s .
Q u e s t i o n s
C o l l e c t i o n
[ / q u e s t i o n s ]
C r e a t e
a
N e w
Q u e s t i o n
[ P O S T ] Y o u
m a y
c r e a t e
y o u r
o w n
q u e s t i o n
u s i n g
t h i s
a c t i o n .
I t
t a k e s
a
J S O N o b j e c t
c o n t a i n i n g
a
q u e s t i o n
a n d
a
c o l l e c t i o n
o f
a n s w e r s
i n
t h e f o r m
o f
c h o i c e s . +
R e q u e s t
( a p p l i c a t i o n / j s o n )
{
" q u e s t i o n " :
" F a v o u r i t e
p r o g r a m m i n g
l a n g u a g e ? " ,
" c h o i c e s " :
[
" S w i f t " ,
" P y t h o n " ,
C " ,
" R u b y "
]
} +
R e s p o n s e
2 0 1
( a p p l i c a t i o n / j s o n )
H e a d e r s
L o c a t i o n :
/ q u e s t i o n s / 2
B o d y
{
" q u e s t i o n " :
" F a v o u r i t e
p r o g r a m m i n g
l a n g u a g e ? " ,
" p u b l i s h e d _ a t " :
0 5 T 0 8 : 4 0 : 5 1 . 6 2 0 Z " ,
" c h o i c e s " :
[
{
" c h o i c e " :
" S w i f t " ,
L e t ' s
g e t
t e s t i n g D R E D D N o
m o r e
o u t d a t e d
A P I
d o c u m e n t a t i o n .
T e s t i n g
o n l y n o d e _ m o d u l e s / . b i n / d r e d d
c m a . a p i b
h t t p s : / / a p i . c o n t e n t f u l . c o m
\
m
G E T
H o o k s b e f o r e A l l
c a l l e d
a t
t h e
b e g i n n i n g
o f
t h e
w h o l e
t e s t
r u n b e f o r e E a c h
c a l l e d
b e f o r e
e a c h
H T T P
t r a n s a c t i o n b e f o r e
c a l l e d
b e f o r e
s o m e
s p e c i f i c
H T T P
t r a n s a c t i o n b e f o r e E a c h V a l i d a t i o n
c a l l e d
b e f o r e
e a c h
H T T P
t r a n s a c t i o n
i s
v a l i d a t e d b e f o r e V a l i d a t i o n
c a l l e d
b e f o r e
s o m e
s p e c i f i c
H T T P
t r a n s a c t i o n
i s
v a l i d a t e d a f t e r
c a l l e d
a � e r
s o m e
s p e c i f i c
H T T P
t r a n s a c t i o n
r e g a r d l e s s
i t s
r e s u l t a f t e r E a c h
c a l l e d
a � e r
e a c h
H T T P
t r a n s a c t i o n a f t e r A l l
c a l l e d
a � e r
w h o l e
t e s t
r u n
H o o k s n o d e _ m o d u l e s / . b i n / d r e d d
c m a . a p i b
h t t p s : / / a p i . c o n t e n t f u l . c o m
\
h o o k s . j s
\
m
G E T
S k i p p i n g
T e s t s v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; h o o k s . b e f o r e (
" W e b h o o k
c a l l s
W e b h o o k
c a l l
d e t a i l s
G e t
t h e
w e b h o o k
c a l l
d e t a i l s " ,
f u n c t i o n
( t r a n s a c t i o n )
{
t r a n s a c t i o n . s k i p
=
t r u e ;
} ) ;
M u t a t i n g
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; h o o k s . b e f o r e (
" E n t r i e s
D e l e t e
a n
E n t r y " ,
f u n c t i o n
( t r a n s a c t i o n )
{
c l i e n t . c r e a t e E n t r y ( e n t r y ,
" 1 2 3 4 " )
} ) ;
R a t e
l i m i t i n g v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; c o n s t
D E L A Y
=
1 0 0 0 / 6 . 0 ; h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
s e t T i m e o u t ( d o n e ,
D E L A Y ) ; } ) ;
C h a n g i n g
r e q u e s t
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; h o o k s . b e f o r e E a c h ( f u n c t i o n
( t r a n s a c t i o n ,
d o n e )
{
t r a n s a c t i o n . f u l l P a t h
=
t r a n s a c t i o n . f u l l P a t h . r e p l a c e ( ' f p 9 1 o e l s z i e a ' ,
' g b k x k l v m o l c 1 ' ) ;
d o n e ( ) ; } ) ;
C h a i
a s s e r t i o n s v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; v a r
a s s e r t
=
r e q u i r e ( ' c h a i ' ) . a s s e r t ; h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
i f
( ! t r a n s a c t i o n . s k i p )
{
i d ' ] ,
9 ] { 2 2 } $ /
}
d o n e ( ) ; } ) ;
S e c u r i t y
�
C e n s o r
p r i v a t e
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; c o n s t
S E C R E T _ H E A D E R S
=
[ ' A u t h o r i z a t i o n ' ] . m a p ( h e a d e r
=
h e a d e r . t o L o w e r C a s e ( ) ) ; h o o k s . a f t e r E a c h V a l i d a t i o n ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
O b j e c t . k e y s ( t r a n s a c t i o n . r e q u e s t . h e a d e r s ) . f o r E a c h ( f u n c t i o n ( k e y )
{
i f
( S E C R E T _ H E A D E R S . i n d e x O f ( k e y . t o L o w e r C a s e ( ) )
1 )
{
t r a n s a c t i o n . r e q u e s t . h e a d e r s [ k e y ]
=
" * * * H I D D E N
S E C R E T
D A T A * * * " ;
}
} ) ;
d o n e ( ) ; } ) ;
C I
t h e
b e a s t
C o n c l u s i o n B a s e
y o u r
d o c u m e n t a t i o n
o n
a n
A P I
s p e c T e s t
t h a t
s p e c M a k e
i t
p a r t
o f
y o u r
C I D o n ' t
s u b s t i t u t e
y o u r
i n t e g r a t i o n
t e s t s
S l i d e s
a v a i l a b l e
o n
S l i d e s h a r e :
d o c u m e n t a t i o n F o l l o w
m e
o n
T w i t t e r :
@ R o u v e n W e s s l i n g
There are few things more frustrating than documentation that doesn’t match the implementation – especially when it’s documentation for your customers. In actively developed software that is a reoccurring problem. As the product evolves the documentation often lags behind and sometimes breaking changes sneak it, breaking the very examples that are part of the documentation. To avoid all these problems, it’s time to start testing your docs. In this talk I’ll show you what’s currently possible when it comes to testing documentation, how to start testing API documentation written using API Blueprints and how to incorporate it into your workflow.
Here’s what was said about this presentation on Twitter.
Automat testing of #API documentation @apiary @dredd doc test dont replace integration test do both @RouvenWessling @contentful #NordicApis
— Alexander Hedlund (@valueinnovate) October 25, 2016
great presentation on API documentation testing by @RouvenWessling "API users rather have no documentation than incorrect documentation"
— Bianca Steinke (@hsv4678) October 25, 2016
Our very own @RouvenWessling talking at @nordicapis right now. #nordicapis @contentful #contentful #contentsocks pic.twitter.com/7WOY353a0X
— Benjamin Knofe (@bknofe) October 25, 2016